{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rent Listing Inqueries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 任务描述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "任务目的： 在Rental Listing Inquiries数据上练习分类方法， 需要根据公寓的特征来预测其受欢迎程度（用户感兴  趣程度分为高、中、低三类）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rental Listing Inquiries数据中，房屋的特征x共有14维，响应值y为用户对该公寓的感兴趣程度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本项目需要：\n",
    "1. 用Logistic回归模型对公寓感兴趣程度进行预测，注意正则超参数的调优。\n",
    "2. 用RBF的SVM对公寓感兴趣程度进行预测，注意正则超参数和核函数参数的调优。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 导入工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.feature_extraction.text import  CountVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "from scipy import sparse\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "from sklearn.cluster import KMeans\n",
    "from nltk.metrics import distance as distance\n",
    "\n",
    "from sklearn import model_selection,preprocessing,ensemble\n",
    "from sklearn.metrics import log_loss\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer,CountVectorizer\n",
    "\n",
    "#from sklearn.cross_validation import StratifiedKFold\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from MeanEncoder import MeanEncoder\n",
    "\n",
    "#可视化\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#import data\n",
    "dpath = \"../RentListingInqueries/\"\n",
    "train = pd.read_json(dpath + \"RentListingInquries_train.json\")\n",
    "test = pd.read_json(dpath + \"RentListingInquries_test.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>created</th>\n",
       "      <th>description</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>interest_level</th>\n",
       "      <th>latitude</th>\n",
       "      <th>listing_id</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>2016-06-24 07:54:24</td>\n",
       "      <td>A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...</td>\n",
       "      <td>Metropolitan Avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>medium</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>7211212</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>[https://photos.renthop.com/2/7211212_1ed4542e...</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10000</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td>2016-06-12 12:19:27</td>\n",
       "      <td></td>\n",
       "      <td>Columbus Avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>low</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>7150865</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>[https://photos.renthop.com/2/7150865_be3306c5...</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100004</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>2016-04-17 03:26:41</td>\n",
       "      <td>Top Top West Village location, beautiful Pre-w...</td>\n",
       "      <td>W 13 Street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>high</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>6887163</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>[https://photos.renthop.com/2/6887163_de85c427...</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100007</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>2016-04-18 02:22:02</td>\n",
       "      <td>Building Amenities - Garage - Garden - fitness...</td>\n",
       "      <td>East 49th Street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>low</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>6888711</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>[https://photos.renthop.com/2/6888711_6e660cee...</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100013</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-04-28 01:32:41</td>\n",
       "      <td>Beautifully renovated 3 bedroom flex 4 bedroom...</td>\n",
       "      <td>West 143rd Street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>low</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>6934781</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>[https://photos.renthop.com/2/6934781_1fa4b41a...</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        bathrooms  bedrooms                       building_id  \\\n",
       "10            1.5         3  53a5b119ba8f7b61d4e010512e0dfc85   \n",
       "10000         1.0         2  c5c8a357cba207596b04d1afd1e4f130   \n",
       "100004        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa   \n",
       "100007        1.0         1  28d9ad350afeaab8027513a3e52ac8d5   \n",
       "100013        1.0         4                                 0   \n",
       "\n",
       "                    created  \\\n",
       "10      2016-06-24 07:54:24   \n",
       "10000   2016-06-12 12:19:27   \n",
       "100004  2016-04-17 03:26:41   \n",
       "100007  2016-04-18 02:22:02   \n",
       "100013  2016-04-28 01:32:41   \n",
       "\n",
       "                                              description  \\\n",
       "10      A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...   \n",
       "10000                                                       \n",
       "100004  Top Top West Village location, beautiful Pre-w...   \n",
       "100007  Building Amenities - Garage - Garden - fitness...   \n",
       "100013  Beautifully renovated 3 bedroom flex 4 bedroom...   \n",
       "\n",
       "            display_address  \\\n",
       "10      Metropolitan Avenue   \n",
       "10000       Columbus Avenue   \n",
       "100004          W 13 Street   \n",
       "100007     East 49th Street   \n",
       "100013    West 143rd Street   \n",
       "\n",
       "                                                 features interest_level  \\\n",
       "10                                                     []         medium   \n",
       "10000   [Doorman, Elevator, Fitness Center, Cats Allow...            low   \n",
       "100004  [Laundry In Building, Dishwasher, Hardwood Flo...           high   \n",
       "100007                          [Hardwood Floors, No Fee]            low   \n",
       "100013                                          [Pre-War]            low   \n",
       "\n",
       "        latitude  listing_id  longitude                        manager_id  \\\n",
       "10       40.7145     7211212   -73.9425  5ba989232d0489da1b5f2c45f6688adc   \n",
       "10000    40.7947     7150865   -73.9667  7533621a882f71e25173b27e3139d83d   \n",
       "100004   40.7388     6887163   -74.0018  d9039c43983f6e564b1482b273bd7b01   \n",
       "100007   40.7539     6888711   -73.9677  1067e078446a7897d2da493d2f741316   \n",
       "100013   40.8241     6934781   -73.9493  98e13ad4b495b9613cef886d79a6291f   \n",
       "\n",
       "                                                   photos  price  \\\n",
       "10      [https://photos.renthop.com/2/7211212_1ed4542e...   3000   \n",
       "10000   [https://photos.renthop.com/2/7150865_be3306c5...   5465   \n",
       "100004  [https://photos.renthop.com/2/6887163_de85c427...   2850   \n",
       "100007  [https://photos.renthop.com/2/6888711_6e660cee...   3275   \n",
       "100013  [https://photos.renthop.com/2/6934781_1fa4b41a...   3350   \n",
       "\n",
       "                 street_address  \n",
       "10      792 Metropolitan Avenue  \n",
       "10000       808 Columbus Avenue  \n",
       "100004          241 W 13 Street  \n",
       "100007     333 East 49th Street  \n",
       "100013    500 West 143rd Street  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 49352 entries, 10 to 99994\n",
      "Data columns (total 15 columns):\n",
      "bathrooms          49352 non-null float64\n",
      "bedrooms           49352 non-null int64\n",
      "building_id        49352 non-null object\n",
      "created            49352 non-null object\n",
      "description        49352 non-null object\n",
      "display_address    49352 non-null object\n",
      "features           49352 non-null object\n",
      "interest_level     49352 non-null object\n",
      "latitude           49352 non-null float64\n",
      "listing_id         49352 non-null int64\n",
      "longitude          49352 non-null float64\n",
      "manager_id         49352 non-null object\n",
      "photos             49352 non-null object\n",
      "price              49352 non-null int64\n",
      "street_address     49352 non-null object\n",
      "dtypes: float64(3), int64(3), object(9)\n",
      "memory usage: 6.0+ MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练数据中空值数量为：  0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Null Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>street_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>photos</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manager_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>longitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>listing_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>latitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>interest_level</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>display_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>description</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>created</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>building_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Null Count\n",
       "street_address            0\n",
       "price                     0\n",
       "photos                    0\n",
       "manager_id                0\n",
       "longitude                 0\n",
       "listing_id                0\n",
       "latitude                  0\n",
       "interest_level            0\n",
       "features                  0\n",
       "display_address           0\n",
       "description               0\n",
       "created                   0\n",
       "building_id               0\n",
       "bedrooms                  0\n",
       "bathrooms                 0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看训练数据哪些属性有缺失值，并统计数量\n",
    "print(\"训练数据中空值数量为： \", train.isnull().values.sum())\n",
    "train_nulls = pd.DataFrame(train.isnull().sum().sort_values(ascending=False)[:20], columns=['Null Count'])\n",
    "train_nulls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comment: 训练集中一共有49352条数据，9个非数值类型属性和6个数值类型属性，数据集中无缺失值。也就意味着，对该数据集不用做数据填补工作，但是需要将非数值类型属性值进行编码，使其成为数值类型属性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 74659 entries, 0 to 99999\n",
      "Data columns (total 14 columns):\n",
      "bathrooms          74659 non-null float64\n",
      "bedrooms           74659 non-null int64\n",
      "building_id        74659 non-null object\n",
      "created            74659 non-null object\n",
      "description        74659 non-null object\n",
      "display_address    74659 non-null object\n",
      "features           74659 non-null object\n",
      "latitude           74659 non-null float64\n",
      "listing_id         74659 non-null int64\n",
      "longitude          74659 non-null float64\n",
      "manager_id         74659 non-null object\n",
      "photos             74659 non-null object\n",
      "price              74659 non-null int64\n",
      "street_address     74659 non-null object\n",
      "dtypes: float64(3), int64(3), object(8)\n",
      "memory usage: 8.5+ MB\n"
     ]
    }
   ],
   "source": [
    "test.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "测试数据中空值数量为：  0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Null Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>street_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>photos</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manager_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>longitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>listing_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>latitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>interest_level</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>display_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>description</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>created</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>building_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Null Count\n",
       "street_address            0\n",
       "price                     0\n",
       "photos                    0\n",
       "manager_id                0\n",
       "longitude                 0\n",
       "listing_id                0\n",
       "latitude                  0\n",
       "interest_level            0\n",
       "features                  0\n",
       "display_address           0\n",
       "description               0\n",
       "created                   0\n",
       "building_id               0\n",
       "bedrooms                  0\n",
       "bathrooms                 0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看测试数据哪些属性有缺失值，并统计数量\n",
    "print(\"测试数据中空值数量为： \", train.isnull().values.sum())\n",
    "test_nulls = pd.DataFrame(train.isnull().sum().sort_values(ascending=False)[:20], columns=['Null Count'])\n",
    "test_nulls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comment: 测试集中一共有74659条数据，8个非数值类型属性和6个数值类型属性，数据集中无缺失值。也就意味着，对该数据集也不用做数据填补工作，但是同上，也需要将非数值类型属性值进行编码，使其成为数值类型属性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 数据探索"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 删除非相关项"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从整体数据来看，照片和id影响不大，直接删除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train.drop(['listing_id','photos'], axis=1, inplace=True)\n",
    "listing_id = test.listing_id.values\n",
    "test.drop(['listing_id','photos'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 interest_level"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x21f8f57db70>"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAELCAYAAAARNxsIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGQpJREFUeJzt3X20XXV95/H3h/DYqgXk6iDBhtHM\nUnQ06hWoWJdPCwIzY9CKwtSSUlajDviwWjuiaxZYlLXqssqIRdbgGIGOFagPQ8aJxgxiEcpDgoZA\nQFZugUqEQjCgoC0U/M4f53flEE5yL2Gfe3KT92uts87e3/3b+/w2Z10+2Xv/zt6pKiRJ6sIuo+6A\nJGnHYahIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOrPrqDsw0/bbb7+aN2/e\nqLshSbPK9ddff19VjU3VbqcLlXnz5rF69epRd0OSZpUk/ziddp7+kiR1ZmihkmTPJNcluSHJuiR/\n3urnJ7k9yZr2WtDqSXJ2kokka5O8sm9bi5Osb6/FffVXJbmxrXN2kgxrfyRJUxvm6a+HgTdW1UNJ\ndgOuTPKttuzPquqrm7U/CpjfXocC5wKHJtkXOB0YBwq4Psmyqrq/tVkCXAMsBxYC30KSNBJDO1Kp\nnofa7G7ttbX77C8CLmzrXQPsnWR/4EhgZVVtakGyEljYlj2rqq6u3v37LwSOGdb+SJKmNtRrKknm\nJFkD3EsvGK5ti85sp7jOSrJHqx0A3Nm3+oZW21p9w4D6oH4sSbI6yeqNGzc+7f2SJA021FCpqseq\nagEwFzgkyUuBjwAvAl4N7At8uDUfdD2ktqE+qB/nVdV4VY2PjU05Ik6StI1mZPRXVT0AfA9YWFV3\nt1NcDwNfAg5pzTYAB/atNhe4a4r63AF1SdKIDHP011iSvdv0XsCbgR+1ayG0kVrHADe1VZYBJ7RR\nYIcBP6uqu4EVwBFJ9kmyD3AEsKItezDJYW1bJwCXDmt/JElTG+bor/2BC5LMoRdel1TVN5N8N8kY\nvdNXa4D3tPbLgaOBCeCXwIkAVbUpyceBVa3dGVW1qU2/Fzgf2IveqC9HfknSCKU3cGrnMT4+Xv6i\nfsf34zP+/ai7sFN4/mk3jroLmiFJrq+q8ana+Yt6SVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlS\nZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcM\nFUlSZwwVSVJnDBVJUmcMFUlSZ4YWKkn2THJdkhuSrEvy561+UJJrk6xPcnGS3Vt9jzY/0ZbP69vW\nR1r91iRH9tUXttpEklOHtS+SpOkZ5pHKw8Abq+rlwAJgYZLDgE8CZ1XVfOB+4KTW/iTg/qp6IXBW\na0eSg4HjgJcAC4HPJ5mTZA5wDnAUcDBwfGsrSRqRoYVK9TzUZndrrwLeCHy11S8AjmnTi9o8bfmb\nkqTVL6qqh6vqdmACOKS9Jqrqtqp6BLiotZUkjchQr6m0I4o1wL3ASuAfgAeq6tHWZANwQJs+ALgT\noC3/GfDs/vpm62ypLkkakaGGSlU9VlULgLn0jixePKhZe88Wlj3V+pMkWZJkdZLVGzdunLrjkqRt\nMiOjv6rqAeB7wGHA3kl2bYvmAne16Q3AgQBt+W8Bm/rrm62zpfqgzz+vqsaranxsbKyLXZIkDTDM\n0V9jSfZu03sBbwZuAS4H3t6aLQYubdPL2jxt+Xerqlr9uDY67CBgPnAdsAqY30aT7U7vYv6yYe2P\nJGlqu07dZJvtD1zQRmntAlxSVd9McjNwUZJPAD8EvtjafxH46yQT9I5QjgOoqnVJLgFuBh4FTq6q\nxwCSnAKsAOYAS6tq3RD3R5I0haGFSlWtBV4xoH4bvesrm9f/BTh2C9s6EzhzQH05sPxpd1aS1Al/\nUS9J6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEi\nSeqMoSJJ6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEiSerM0EIlyYFJLk9y\nS5J1ST7Q6h9L8pMka9rr6L51PpJkIsmtSY7sqy9stYkkp/bVD0pybZL1SS5Osvuw9keSNLVhHqk8\nCvxpVb0YOAw4OcnBbdlZVbWgvZYDtGXHAS8BFgKfTzInyRzgHOAo4GDg+L7tfLJtaz5wP3DSEPdH\nkjSFoYVKVd1dVT9o0w8CtwAHbGWVRcBFVfVwVd0OTACHtNdEVd1WVY8AFwGLkgR4I/DVtv4FwDHD\n2RtJ0nTMyDWVJPOAVwDXttIpSdYmWZpkn1Y7ALizb7UNrbal+rOBB6rq0c3qkqQRGXqoJHkG8DXg\ng1X1c+Bc4AXAAuBu4NOTTQesXttQH9SHJUlWJ1m9cePGp7gHkqTpGmqoJNmNXqB8uaq+DlBV91TV\nY1X1K+AL9E5vQe9I48C+1ecCd22lfh+wd5JdN6s/SVWdV1XjVTU+NjbWzc5Jkp5kmKO/AnwRuKWq\nPtNX37+v2VuBm9r0MuC4JHskOQiYD1wHrALmt5Feu9O7mL+sqgq4HHh7W38xcOmw9keSNLVdp26y\nzQ4H/gC4McmaVvsovdFbC+idqroDeDdAVa1LcglwM72RYydX1WMASU4BVgBzgKVVta5t78PARUk+\nAfyQXohJkkZkaKFSVVcy+LrH8q2scyZw5oD68kHrVdVtPH76TJI0Yv6iXpLUGUNFktQZQ0WS1BlD\nRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS\n1BlDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS1JmhhUqSA5NcnuSWJOuSfKDV902yMsn69r5P\nqyfJ2UkmkqxN8sq+bS1u7dcnWdxXf1WSG9s6ZyfJsPZHkjS1YR6pPAr8aVW9GDgMODnJwcCpwGVV\nNR+4rM0DHAXMb68lwLnQCyHgdOBQ4BDg9Mkgam2W9K23cIj7I0mawtBCparurqoftOkHgVuAA4BF\nwAWt2QXAMW16EXBh9VwD7J1kf+BIYGVVbaqq+4GVwMK27FlVdXVVFXBh37YkSSMwI9dUkswDXgFc\nCzy3qu6GXvAAz2nNDgDu7FttQ6ttrb5hQF2SNCLTCpUkl02ntoV1nwF8DfhgVf18a00H1Gob6oP6\nsCTJ6iSrN27cOFWXJUnbaKuhkmTPdk1jvyT7tIvs+7Yjj+dNtfEku9ELlC9X1ddb+Z526or2fm+r\nbwAO7Ft9LnDXFPW5A+pPUlXnVdV4VY2PjY1N1W1J0jaa6kjl3cD1wIva++TrUuCcra3YRmJ9Ebil\nqj7Tt2gZMDmCa3Hb1mT9hDYK7DDgZ+302ArgiBZq+wBHACvasgeTHNY+64S+bUmSRmDXrS2sqs8C\nn03yvqr63FPc9uHAHwA3JlnTah8F/gK4JMlJwI+BY9uy5cDRwATwS+DE1odNST4OrGrtzqiqTW36\nvcD5wF7At9pLkjQiWw2VSVX1uSSvAeb1r1NVF25lnSsZfN0D4E0D2hdw8ha2tRRYOqC+Gnjp1vou\nSZo50wqVJH8NvABYAzzWypPDeCVJAqYZKsA4cHA7mpAkaaDp/k7lJuDfDLMjkqTZb7pHKvsBNye5\nDnh4slhVbxlKryRJs9J0Q+Vjw+yEJGnHMN3RX3837I5Ikma/6Y7+epDHb4GyO7Ab8IuqetawOiZJ\nmn2me6TyzP75JMfQuw29JEm/tk13Ka6q/w28seO+SJJmueme/npb3+wu9H634m9WJElPMN3RX/+p\nb/pR4A56D9WSJOnXpntN5cRhd0SSNPtN9yFdc5N8I8m9Se5J8rUkc6deU5K0M5nuhfov0XveyfPo\nPbL3/7SaJEm/Nt1QGauqL1XVo+11PuAjFCVJTzDdULkvybuSzGmvdwE/HWbHJEmzz3RD5Y+AdwD/\nBNwNvJ32ZEZJkiZNd0jxx4HFVXU/QJJ9gb+kFzaSJAHTP1J52WSgQO+58cArhtMlSdJsNd1Q2SXJ\nPpMz7Uhlukc5kqSdxHSD4dPA3yf5Kr3bs7wDOHNovZIkzUrT/UX9hUlW07uJZIC3VdXNQ+2ZJGnW\nmfZdiqvq5qr6q6r63HQCJcnS9gv8m/pqH0vykyRr2uvovmUfSTKR5NYkR/bVF7baRJJT++oHJbk2\nyfokFyfZfbr7Ikkajm269f00nQ8sHFA/q6oWtNdygCQHA8cBL2nrfH7yNzHAOcBRwMHA8a0twCfb\ntuYD9wMnDXFfJEnTMLRQqaorgE3TbL4IuKiqHq6q24EJeg8BOwSYqKrbquoR4CJgUZLQOxX31bb+\nBcAxne6AJOkpG+aRypackmRtOz02OaLsAODOvjYbWm1L9WcDD1TVo5vVB0qyJMnqJKs3btzY1X5I\nkjYz06FyLvACYAG9X+Z/utUzoG1tQ32gqjqvqsaranxszFuWSdKwzOhvTarqnsnpJF8AvtlmNwAH\n9jWdC9zVpgfV7wP2TrJrO1rpby9JGpEZPVJJsn/f7FuByZFhy4DjkuyR5CBgPnAdsAqY30Z67U7v\nYv6yqirgcnr3IANYDFw6E/sgSdqyoR2pJPkK8HpgvyQbgNOB1ydZQO9U1R3AuwGqal2SS4Cb6T2u\n+OSqeqxt5xRgBTAHWFpV69pHfBi4KMkngB8CXxzWvkiSpmdooVJVxw8ob/F//FV1JgN+pd+GHS8f\nUL+N3ugwSdJ2YhSjvyRJOyhDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZ\nQ0WS1BlDRZLUGUNFktQZQ0WS1BlDRZLUGUNFktQZQ0WS1BlDRZLUmaE9+XFH8Ko/u3DUXdjhXf+p\nE0bdBUkd8khFktQZQ0WS1BlDRZLUmaGFSpKlSe5NclNfbd8kK5Osb+/7tHqSnJ1kIsnaJK/sW2dx\na78+yeK++quS3NjWOTtJhrUvkqTpGeaRyvnAws1qpwKXVdV84LI2D3AUML+9lgDnQi+EgNOBQ4FD\ngNMng6i1WdK33uafJUmaYUMLlaq6Ati0WXkRcEGbvgA4pq9+YfVcA+ydZH/gSGBlVW2qqvuBlcDC\ntuxZVXV1VRVwYd+2JEkjMtPXVJ5bVXcDtPfntPoBwJ197Ta02tbqGwbUB0qyJMnqJKs3btz4tHdC\nkjTY9nKhftD1kNqG+kBVdV5VjVfV+NjY2DZ2UZI0lZkOlXvaqSva+72tvgE4sK/dXOCuKepzB9Ql\nSSM006GyDJgcwbUYuLSvfkIbBXYY8LN2emwFcESSfdoF+iOAFW3Zg0kOa6O+TujbliRpRIZ2m5Yk\nXwFeD+yXZAO9UVx/AVyS5CTgx8Cxrfly4GhgAvglcCJAVW1K8nFgVWt3RlVNXvx/L70RZnsB32ov\nSdIIDS1Uqur4LSx604C2BZy8he0sBZYOqK8GXvp0+ihJ6tb2cqFekrQDMFQkSZ0xVCRJnTFUJEmd\nMVQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnTFU\nJEmdMVQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnRlJqCS5I8mNSdYkWd1q+yZZmWR9e9+n1ZPk7CQT\nSdYmeWXfdha39uuTLB7FvkiSHjfKI5U3VNWCqhpv86cCl1XVfOCyNg9wFDC/vZYA50IvhIDTgUOB\nQ4DTJ4NIkjQa29Ppr0XABW36AuCYvvqF1XMNsHeS/YEjgZVVtamq7gdWAgtnutOSpMeNKlQK+E6S\n65MsabXnVtXdAO39Oa1+AHBn37obWm1LdUnSiOw6os89vKruSvIcYGWSH22lbQbUaiv1J2+gF1xL\nAJ7//Oc/1b5KkqZpJEcqVXVXe78X+Aa9ayL3tNNatPd7W/MNwIF9q88F7tpKfdDnnVdV41U1PjY2\n1uWuSJL6zHioJPnNJM+cnAaOAG4ClgGTI7gWA5e26WXACW0U2GHAz9rpsRXAEUn2aRfoj2g1SdKI\njOL013OBbySZ/Py/qapvJ1kFXJLkJODHwLGt/XLgaGAC+CVwIkBVbUrycWBVa3dGVW2aud2QJG1u\nxkOlqm4DXj6g/lPgTQPqBZy8hW0tBZZ23UdJ0rbZnoYUS5JmOUNFktSZUQ0plqQtOvxzh4+6Czu8\nq9531VC265GKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkz\nhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM7M+VJIsTHJrkokkp466P5K0\nM5vVoZJkDnAOcBRwMHB8koNH2ytJ2nnN6lABDgEmquq2qnoEuAhYNOI+SdJOa7aHygHAnX3zG1pN\nkjQCu466A09TBtTqSY2SJcCSNvtQkluH2qvR2g+4b9SdmK785eJRd2F7Mqu+OwBOH/QnuNOaVd9f\n3v+Uv7vfnk6j2R4qG4AD++bnAndt3qiqzgPOm6lOjVKS1VU1Pup+6Knzu5vd/P56Zvvpr1XA/CQH\nJdkdOA5YNuI+SdJOa1YfqVTVo0lOAVYAc4ClVbVuxN2SpJ3WrA4VgKpaDiwfdT+2IzvFab4dlN/d\n7Ob3B6TqSde1JUnaJrP9mookaTtiqOxAknwvyXibXp5k71H3SU+U5KFR90FPTZJ5SW4aUD8jyZun\nWPdjST40vN5tf2b9NRUNVlVHj7oP0o6sqk4bdR+2Rx6pjFj7V9CPkvzPJDcl+XKSNye5Ksn6JIck\n+c0kS5OsSvLDJIvaunsluSjJ2iQXA3v1bfeOJPtt/q+sJB9K8rE2/b0kZyW5IsktSV6d5Ovtcz8x\n0/8tdibp+VT7zm9M8s5W/3ySt7TpbyRZ2qZP8jsZqTlJvpBkXZLvtL+985O8HSDJ0e3v+MokZyf5\nZt+6B7e/tduSvH9E/Z8xHqlsH14IHEvvV/+rgP8MvBZ4C/BR4Gbgu1X1R+2U1nVJ/h/wbuCXVfWy\nJC8DfrANn/1IVb0uyQeAS4FXAZuAf0hyVlX99OnunAZ6G7AAeDm9X2KvSnIFcAXwu/R+b3UAsH9r\n/1p697bTaMwHjq+qP05yCfB7kwuS7An8D+B1VXV7kq9stu6LgDcAzwRuTXJuVf3rTHV8pnmksn24\nvapurKpfAeuAy6o3LO9GYB5wBHBqkjXA94A9gecDrwP+F0BVrQXWbsNnT/5Y9EZgXVXdXVUPA7fx\nxLsVqFuvBb5SVY9V1T3A3wGvBr4P/G672/bNwD1J9gd+B/j7kfVWt1fVmjZ9Pb2/y0kvAm6rqtvb\n/Oah8n+r6uGqug+4F3juUHs6Yh6pbB8e7pv+Vd/8r+h9R48Bv1dVT7hnWRIYcK+zzTzKE//xsOcW\nPrv/c/s/W8Mx8MZLVfWTJPsAC+kdtewLvAN4qKoenMH+6Yn6/zYeo+9UM1v4Lrey7g79d+WRyuyw\nAnhfWookeUWrXwH8fqu9FHjZgHXvAZ6T5NlJ9gD+4wz0V1O7AnhnkjlJxugddV7Xll0NfLC1+T7w\nofau7dOPgH+bZF6bf+foujJ6O3Ri7kA+Dvx3YG0LljvohcO5wJeSrAXW8Pj/lH6tqv41yRnAtcDt\n9P4ANHrfoHdK6wZ6R5v/tar+qS37PnBEVU0k+Ud6RyuGynaqqv45yX8Bvp3kPgb8He5M/EW9JD1N\nSZ5RVQ+1f/SdA6yvqrNG3a9R8PSXJD19f9wG0qwDfoveaLCdkkcqkqTOeKQiSeqMoSJJ6oyhIknq\njKEiSeqMoSIBSaa8BUqSDyb5jSH3Y0GSrd5hOskfJvmrjj+3821q52SoSEBVvWYazT4IPKVQSTLn\nKXZlAeBjCzRrGSoSjz88K8nr223Kv9puZf7ldpv69wPPAy5Pcnlre0SSq5P8IMnfJnlGq9+R5LQk\nVwLHJnlBkm8nuT7J95O8qLU7tt36/ob2+IHdgTPo3b5lzeTt8Kfo91iSr7XHIqxKcniSXVof9u5r\nN5HkuYPad/4fUzs1b9MiPdkrgJcAdwFXAYdX1dlJ/gR4Q1Xdl2Q/4L8Bb66qXyT5MPAn9EIB4F+q\n6rUASS4D3lNV65McCnweeCNwGnBku4nk3lX1SJLTgPGqOmWaff0scFZVXZnk+cCKqnpxkkuBt9K7\njc+hwB1VdU+Sv9m8PfDip/nfS/o1Q0V6suuqagNA+5X0PODKzdocBhwMXNXu87k7vRtBTrq4rf8M\n4DXA37Z2AHu096uA89vzOb6+jX19M72HQE3OPyvJM9vnnwZ8CThusj9baS91wlCRnmw6tyoPsLKq\njt/CNn7R3ncBHqiqBZs3qKr3tKOI/wCsSfKkNtOwC/A7VfXPT+hccjXwwnYH5GOAT0zRfhs+Wnoy\nr6lI0/cgvaf3AVwDHJ7khQBJfiPJv9t8har6OXB7kmNbuyR5eZt+QVVd2551fh+9h6L1f8Z0fAf4\n9amyyWBqD3n7BvAZ4Ja+J3gObC91xVCRpu884FtJLq+qjcAfAl9pjx64ht4TAAf5feCkJDfQu+Hg\nolb/VHrPp7+J3rNTbgAup3d6aloX6oH3A+NJ1ia5GXhP37KLgXfx+KmvqdpLT5s3lJQkdcYjFUlS\nZ7xQL22nkpwIfGCz8lVVdfIo+iNNh6e/JEmd8fSXJKkzhookqTOGiiSpM4aKJKkzhookqTP/H0Mg\nXhgvx4dVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f4a992ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(train.interest_level)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "兴趣分布大多为low，其次是medium， 最后是high  \n",
    "#### 对interest_level编码，对应0,1,2三个等级"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10        medium\n",
       "10000        low\n",
       "100004      high\n",
       "100007       low\n",
       "100013       low\n",
       "100014    medium\n",
       "100016       low\n",
       "100020       low\n",
       "100026    medium\n",
       "100027       low\n",
       "100030       low\n",
       "10004        low\n",
       "100044      high\n",
       "100048       low\n",
       "10005        low\n",
       "100051    medium\n",
       "100052       low\n",
       "100053       low\n",
       "100055       low\n",
       "100058       low\n",
       "100062       low\n",
       "100063    medium\n",
       "100065       low\n",
       "100066       low\n",
       "10007     medium\n",
       "100071       low\n",
       "100075    medium\n",
       "100076       low\n",
       "100079      high\n",
       "100081       low\n",
       "           ...  \n",
       "99915        low\n",
       "99917        low\n",
       "99919     medium\n",
       "99921     medium\n",
       "99923        low\n",
       "99924        low\n",
       "99931        low\n",
       "99933        low\n",
       "99935        low\n",
       "99937        low\n",
       "9994         low\n",
       "99953        low\n",
       "99956        low\n",
       "99960     medium\n",
       "99961        low\n",
       "99964     medium\n",
       "99965        low\n",
       "99966        low\n",
       "99979        low\n",
       "99980        low\n",
       "99982       high\n",
       "99984        low\n",
       "99986        low\n",
       "99987        low\n",
       "99988     medium\n",
       "9999      medium\n",
       "99991        low\n",
       "99992        low\n",
       "99993        low\n",
       "99994        low\n",
       "Name: interest_level, Length: 49352, dtype: object"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train['interest_level']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_map = {'low': 0, 'medium': 1, 'high': 2}\n",
    "train['interest_level'] = train['interest_level'].apply(lambda x: y_map[x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 bathrooms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "49352\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAELCAYAAAA2mZrgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8XOWZ9//PNaPeLBtJLuruBRuw\nBQQI/AzkIcbxhmyAXbJsgNACCTE8IQmQ3QcCS0uyQEJMWSB9CSSbCl6TBBJ6l40L7ipWsY0l25LV\nNe36/TGjkSxka1RGZ8a63q/XvHRmdGbmkmbO+Z77nPvcR1QVY4wxBsDldAHGGGNih4WCMcaYMAsF\nY4wxYRYKxhhjwiwUjDHGhFkoGGOMCbNQMMYYE2ahYIwxJsxCwRhjTFiC0wUMVU5OjpaUlDhdhjHG\nxJW1a9fuV9XcweaLu1AoKSmhvLzc6TKMMSauiEhNJPPZ7iNjjDFhFgrGGGPCLBSMMcaEWSgYY4wJ\ns1AwxhgTZqFgjDEmzELBGGNMmIWCMcaYMAuFGOLz+Xj77bfx+XxOl2KG4Q9/+AOPPvoodt3z+OPx\neLj66qt54403nC7FcRYKMeTFF1/klltu4aWXXnK6FDMMDz30EM8++ywNDQ1Ol2KGqKmpiR07drBq\n1SqnS3GchUIM2bt372E/TfxobW0NT9fX1ztYiRmOQCAAQFdXl8OVOM9CIQbZ7of4U1lZOeC0iQ8e\njwcAEXG4EudFLRREJEVE3hORDSKyWUTuHGCeZBH5tYhUiMi7IlISrXriiX0x48+WLVsASHbD5s2b\nHa7GDFVPKNgGWXRbCt3AOap6AnAisExEPtFvnquAJlWdCTwEfDeK9cS8ni+kfTHjz7Zt28hNgxOO\n87Bti4VCvOnu7gZsgwyiGAoa1Ba6mxi69V/bXQD8PDT9W+BcGcefSs9+TQuF+LN96xZKMzyUZPnY\nu6/hsGMMJvZZS6FXVI8piIhbRNYDDcCLqvpuv1nygToAVfUBh4DjollTLPN6vUDvF9TEB1VlX+N+\npqT5mZoWDPZ9+/Y5XJUZip5u4C6XHWaN6n9AVf2qeiJQAJwiIsf3m2WgVsHHolpErhWRchEpb2xs\njEapMaGnpdDz08SH7u5uAoEAaQlKqjv49W1vb3e4KjMU1krvNSaxqKrNwCvAsn6/qgcKAUQkAZgA\nHBzg+U+oapmqluXmDno1OWPGlMvlQkTwBII3gMTERIerMkPR00IYx3uvw6LZ+yhXRLJD06nAp4Bt\n/WZ7Drg8NH0R8Hcdx1GdnJwMQFJSksOVmKFISkriuEnZNHa6aOwMLlJTp051uCozFG63G7CWAkT3\nGs1TgZ+LiJtg+PxGVVeLyF1Auao+B/wY+KWIVBBsIVwSxXpiXs8Xs+eniR/FJdOp3XEAl0BmRjrZ\n2dlOl2SGwDbEekUtFFR1I3DSAI/f3me6C7g4WjXEK2vCxp8FCxbwy3VrafclseCEhfYZxpmeULCW\ngp3RHJPsixl/5s2bhyrs7xTmzp3rdDlmiHp23VqYWyjEJPtixp/Zs2cPOG3ig7UUelkoxCD7Ysaf\nnJyc8HRxcbGDlZjhSEgI7km3DTILhZhkX8z40/czy8vLc7ASMxw9XVJtg8xCISbZFzO+9eyfNvHH\nNsgsFGKSfTGNGVu2IdbLQiEG2RfUGOMUC4UYZC0FY8ZWzzJnG2QWCjHJvpjGOMM2yCwUjDHG9GGh\nYIwxJsxCwRhjTJiFgjHGmDALBWOMMWEWCsYYY8IsFIwxxoRZKBhjjAmzUDDGGBNmoWCMMSbMQiEG\n2an2xhinWCjEIBv7KL4FAgGnSzBm2CwUYpC1FOJbR0eH0yUYM2xRCwURKRSRl0Vkq4hsFpEbB5hn\nqYgcEpH1odvt0aonnlhLIb61t7c7XYIxw5YQxdf2ATer6joRyQTWisiLqrql33yvq+qKKNYRd6yl\nEH/6BnlbWxuTJ092sBpjhi9qLQVV3auq60LTrcBWID9a73cssZZC/PH5fOHprq4uBysxZmTG5JiC\niJQAJwHvDvDr00Rkg4i8ICILjvD8a0WkXETKGxsbo1hpbLCWQvzxeDwDThsTb6IeCiKSAfwOuElV\nW/r9eh1QrKonAD8C/jjQa6jqE6papqplubm50S04BlhLIf70DXILdRPPohoKIpJIMBCeVtXf9/+9\nqraoaltoeg2QKCI50awpHthKJf4kJiaGpxMSonmozpjoimbvIwF+DGxV1QePMM+U0HyIyCmheg5E\nq6Z4YS2F+NM3CFJTUx2sxJiRieYmzRnAF4FNIrI+9Ni3gSIAVX0cuAi4XkR8QCdwidoa0VoKcajv\nZ5aRkeFgJcaMTNRCQVXfAI66dlPVVcCqaNUQrywX45uFgolndkZzDLKWQnyz3UcmnlkoxCBrKcQ3\nl8sWKxO/7Nsbg6ylYIxxioVCDLKWgjHGKRYKxhhjwiwUjDHGhFkoGGOMCbNQMMYYE2ahYIwZ96xz\nRy8LBWOMCbHu4BYKxhgTZi0GCwVjjAmzloKFQkyxrRRjnNETBrYMWigYYwxer9fpEmKGhUIMsa2U\nY4PP53O6BDNEHR0dgC2DYKEQUzo7O4HeL6iJH11dXeHpHTt2OFiJGY66ujoA/H6/w5U4z0IhhvSs\nTGylEn/WrVsXnn733XcdrMQMx86dOwHweDwEAgGHq3GWhUKMWL9+PRs2bEDFzfr161m/fv3gTzIx\nwePx8Ogjq8hNgwWTfDz7zK9oaGhwuiwToUAgwJ//8mcg2FovLy93uCJnWSjEgJaWFh544EFIzqDj\npC9AcgYPPvgQra2tTpdmBhEIBFi1ahW1dfVcPruVq+a14/N2c889d4d3B5rY9uqrr9LY0EjglACS\nIvzqV78a18cWLBQctn//fr56ww3U1tXRVXIGJKbQVXIGNbW1fPWGG9i/f7/TJZoj8Hq93HPP3fzx\nj39kWVEXJ+b4yEsN8KU57WxYv56bbrqRQ4cOOV2mOYqmpiYeePABmAhaqPjn+Vm3bh3PP/+806U5\nxkLBQRUVFVx//VeordtNx+zz8GcXAuDPLqRj9nnU1NZz/fVfoaKiwuFKTX/Nzc1865vf5MUXX+Kf\nZnRy6azeVsFZ0zysXNhGxfZtfOW666iurnawUnMkfr+fe+69h9a2Vvwn+8EFOkNhMvzoRz9i165d\nTpfoiKiFgogUisjLIrJVRDaLyI0DzCMi8rCIVIjIRhFZHK16Ys2aNWv48pevo6G5lY655xOYkH/Y\n7wMT8umYez4Nza18+cvX8cILLzhUqelv06ZNXPmlK9i44QOund/OZ0u76H8ibFmel1sXt9Cyfzdf\nvvYa/vrXvzpTrDmin/70p7z37nv4T/DDhNCDAv6T/XhcHm697Vba2tocrdEJ0Wwp+ICbVXUe8Ang\nqyIyv9885wOzQrdrgceiWE/MePTRR7n//vvpSs2hbcEFBDLyBpwvkJFH24IL6Eo9jvvuu4/HHhsX\n/56Y9uabb7Jy5ddwdx7kO2WHOGua54jzzsn2c/cpzZSkdXL33Xfzy1/+cgwrNUezfv16fvGLXxAo\nCaDT+x0/SAXfqT727N3DQw895EyBDopaKKjqXlVdF5puBbYC+f1muwD4hQa9A2SLyNRo1RQL/vrX\nv/Lss8/izZtL19xlkJh29CckptE193y8eXN55plnePHFF8emUPMxtbW1/Mddd1KU7uOuk5spzhy8\nT/vEZOW2k1o4fUo3Tz75JG+99dYYVGoG89RTTyFpgi5WGGi4o1wIzArw0ksvjbvdSGNyTEFESoCT\ngP4duPOBuj736/l4cBwzmpub+e73vkcgayqektNBIvz3iwtP8ekEsqby3e99j+bm5ugWagb0ve/e\nj9vfxY2LWkhPjLx3itsFV8/roCQrwH/cdacNqeCwiooKNm7ciH+WH9xHnk/nKLjh97///dgVFwOi\nHgoikgH8DrhJVVv6/3qAp3xsaRORa0WkXETKGxsbo1HmmFBVfF4vvswpkQdCD5cLX+YUvB7PuO4u\n5yQFclMD5KQM/f+f5IbpmV5U1UbidFj4/586yIxJgIDLNb7640T1rxWRRIKB8LSqDhS39UBhn/sF\nwJ7+M6nqE6papqplubm50Sl2DEycOJGFCxeSdKAS8QxtKAvxdJB0oIJFixYxceLEKFVojqas7GSq\nD7nY3nyUzcsjaOx0sW5/MieetJiEhIQoVGciNW3atODEgUFmbAX1au/840Q0ex8J8GNgq6o+eITZ\nngMuC/VC+gRwSFX3RqumWPClL32JJO0mbevzSMfBiJ4jHQdI2/o8SerhiiuuiG6B5oj+4R/+gYL8\nfO5fl8V7+xIjfl51i5s7107Al5DGZZddFsUKTSRSU1M577zzcFW6oOkIMym417pJS0/jnHPOGdP6\nnBbNlsIZwBeBc0Rkfei2XESuE5HrQvOsAaqACuBJ4CtRrCcmLFmyhEdWrWJiWhLpW5/H3VRz1Pnd\nB2tI37qaSelJPPrIIyxZsmSMKjX95eTk8OjjjzNn3nx+tCmDP1WnEBhkT9J7+xK5e90EkjJzePSx\nx5k/v38HPOOElStXMjF7IgnvJwT7SfYj2wT2w/+96f+Sk5Mz9gU6SOJt/3RZWZkeC2OTNDY2cttt\n32bHju148k/Cm7+Ywzq7q5K4ex1Juz9g9uw53HffvcTzrrNjSXd3N/fffz9/+9vfWJLr5csL2kjr\nt0fIH4DfVKbyvzUpzJs3l3vuuXfcrVxiXXl5OTfffDP+fD/6iT69kD4C9+tuzjnnHO64445j5hiQ\niKxV1bLB5htyS0FEXCKSNbyyTI/c3FweeWQVy5YtI2n3ByRXvgo9Aa1KcuWrJO3+gGXLlvHII6ss\nEGJIcnIyt99+OytXrmT9wWTuLM+m1dO74vAH4IebMvjfmhQuuOACfvSjVRYIMaisrIzrrrsOV70L\nqQp9ft2Q8F4CpdNLueWWW46ZQBiKiEJBRH4lIlkikg5sAbaLyDejW9qxLzk5mdtuu40rr7yShAMV\nJHy0CYDEvZtIOFDBlVdeyW233UZycrLDlZr+RISLLrqI//zPB2joSuChjZl4/MFc/9n2NNY1JnLj\njTdy8803k5SU5HS55gguueQSTjzpRNyb3eAB+VAQr3Dnd+4kNXWw7knHpkhbCvND3Uk/R/A4QBHB\n4wVmhESEyy+/nKVLl5Jc9z4JDdtIqn+fs88+m8svv3xcbqnEkyVLlvDtf/t3djS7eWZnKq/vTeLl\n3clceumlXHjhhU6XZwYhIty48sZgIKwTXNUuPv/5z1NSUuJ0aY6JNBQSQ91LPwf8SVW9DHA+gRke\nEeGWW24hMTGR5Oo3SExM5Fvf+pYFQpw499xzWb58Oa/uTeW3VWnMnDGDa665xumyTIRmzJjB4sWL\ncdW5QOHiiy92uiRHRRoK/wXsAtKB10SkGOh/IpoZgfT0dM44/XQAzjj9dNLT0x2uyAzF5z//eTx+\n5WCX8PkLLxx3JzzFu9NOOw0IdledOvWYHmlnUBGdRaOqDwMP93moRkTOjk5J41dxcTHAuG66xqvZ\ns2eHp08++WQHKzHDMWPGDIBxexyhr4hCQUSygcuAkn7PWRmFmoyJa9bTKP7YZ9Yr0vPt1wDvAJuA\n8X1V6yjqOYZgxxLim9s99GEwjLN6Wgi27EUeCimq+vWoVmLC4u2EQmPinYVBr0iPhv1SRK4Rkaki\nMqnnFtXKxjH7ghpjnBJpS8EDfB/4N3q7oiowPRpFjXfWUjBmbPVsiNmyF3kofB2Yqar7o1mMMcY4\nyVrpke8+2gwM7QIAxhhj4k6kLQU/sF5EXga6ex5UVeuSGgW2tWKMcUqkofDH0M2MAduvaYxxSqRn\nNP9cRJKAntM2t4fGPzJRYC0FY4xTIj2jeSnwc4LjHwlQKCKXq+pr0Stt/LKWgjHGKZHuPnoAOE9V\ntwOIyGzgGcCuDRkF1lIwxjgl4qGzewIBQFV3AJFfudwMibUU4pvf73e6BGOGLdJQKBeRH4vI0tDt\nSWBtNAsbjywMjg1NTU1Ol2DMsEUaCtcTPFdhJXAjwUtyXhetosYr220UvzweT3i6oaHBwUqMGZmI\nQkFVu4FVwB3A/wNWhR47IhH5iYg0iMiHR/j9UhE5JCLrQ7fbh1r8scrCIf7s3bs3PL17924HKzFm\nZKLZ++hnBIPkF0eZ53VVXRFRpeOI7UaKP32DwELBxLOo9T5S1ddEpGSkBY5H1lKIP313GdnuIxPP\nnO59dJqIbBCRF0RkwSi83jHBWgrx58CBAwhQlBlg/34bN9LEr0hbCuUi8mPgl6H7lzLy3kfrgGJV\nbROR5QSH0Zg10Iwici1wLUBRUdEI39aY0efz+XC7INml+Hw+p8sxZtgc632kqi2q2haaXgMkisiA\nF0pV1SdUtUxVy3Jzc0fytsZEhYigGrxWre3+M/Fs0JaCiLiBH6vqvwIPjtYbi8gUYJ+qqoicQjCg\nDozW68cj220UvyZMmIBf4aMON9Ozs50ux5hhGzQUVNUvIrkikqSqnsHm7yEizwBLgRwRqSfYnTUx\n9JqPAxcB14uID+gELtFxvla0Lcz41dOCbfdCTs6ADV5j4kKkxxR2AW+KyHNAe8+DqnrEloOqfuFo\nL6iqqwh2WTUh4zwT49q0adPC0/n5+Q5WYszIRBoKe0I3F5AZvXKMiU99g8BCwcSzSK+ncGe0CzEm\nnmVm9m4rTZ482cFKjBmZSM9ong18Ayjp+xxVPSc6ZY1Pdkzh2GDHFEw8i3T30f8AjwNPEbxes4ki\nC4f4lpKS4nQJxgxbpKHgU9XHolqJCbMDzsYYpxw1FERkUmjyeRH5CvAHIDw6qqoejGJt45a1FIwx\nThmspbAWUIIjowJ8s8/vFJgejaLGO2spGGOcctRQUNVSABFJUdWuvr8TEdtxGiXWUjDGOCXSsY/e\nivAxMwqspWCMM2zZG/yYwhQgH0gVkZPo3Y2UBaRFubZxy1oKxjjDlr3Bjyl8GrgCKODwwfBagW9H\nqaZxz7ZWjDFOGeyYws+Bn4vIhar6uzGqadyzrRVjxlbPNTBs2Yt8mIvfichngAVASp/H74pWYeNR\nTwvBWgrGjK2WlhYAu0ASER5oFpHHgX8GvkbwuMLFQHEU6xqXAoEAYKEQ75qbm50uwQzRli1bAPB4\nIr46wDEr0t5Hp6vqZUBTaHC804DC6JU1PtXW1gKwa9cuZwsxQ1ZZWRmefvvttx2sxAxVV1cXz69+\nHoDOzk7Wr1/vcEXOijQUOkM/O0RkGuAFSqNT0vi0ceNGXn31VQBeffVVNm3a5HBFJlJer5d77v4P\nMpJgclqAVT96mP379ztdlomAz+fj9ttvZ+eOnQSWBCATbrn1lsNCfryJNBRWi0g28D2CZznvAp6N\nVlHjSWtrK08++SQ33/wNJDmD9sWXIskZfP3rN/PUU0/R2trqdInmCAKBAK+88gpXX3UlFZVVXD23\njZtPaKWrs51rrr6K3/3ud3R3dw/+QmbMNTc388wzz/Avl/4L77zzDoHFAXS64vukjy66uPbL13Lf\nffexZcuWcbc7VyL5g0UkFbgeOJPg8BavA4/1P8t5LJSVlWl5eflYv+2oO3DgAC+88AL//fTTdLS3\n4ztuOp6CMjQlC+lqIamunISDVaSnZ3Dppf/C8uXLmTRp0uAvbKLO6/Xy9ttv89Of/JjKqmqmpisX\nTW/n1MleAHY2u/l1ZRrbmhLIOW4S//rFy1i2bBlpaXZqj5P8fj+bN2/mT3/6Ey+//HLwoHIu+Gf6\ng53ue7SDbBPctW7Up8yYOYN//Nw/cu6555Kenu5Y/SMlImtVtWzQ+SIMhd8QPDfhv0MPfQHIVtV/\nGlGVwxCPoeDz+aiqqmLz5s18+OGHbNi4iYZ9HwHgzy7CU7CEQPpxH3ueq/0ASfXluJvrAJg8ZSqL\nFh7P8ccHb6WlpSQkRDrQrRmujo4OtmzZwsaNG9mwYT1bNm+m2+NlSrryuZIOTp/iwdWvJ6MqbG5K\n4PdVaexoduNyuZg9ayaLTjiRE044gYULF5Kdne3MHzQOqCp79+5l27ZtbNu2ja1bt7Jt+za6u7qR\nJMFf5EenK0w4yot4QWoEd5UbPaSICIVFhcyfN5+5c+cyd+5cZsyYQXJy8pj9XSMx2qGwQVVPGOyx\nsRDroeD3+9m3bx81NTV8+OGHfPjhZrZs2UJ3d7BRJcnpeNNy8WfmEciaRiB98AuyuNr342rZg7u1\ngcSOBrS7A4Dk5BTmz5/P8ccv4Pjjj6e4uJjJkyfjdruj+jcey1SVAwcOsHXrVjZs2MDGDevZsbOC\nQCCACBRnBpgzwcP8iT5OzPHiHmQHrCpsb05g04EEth9KpPJQAt5gJzOKiwrDIbFgwQKmTp2KyxXp\nHl3TQ1XZv38/27dvDwbAtq1s3bqVttY2AMQlaLYSmBiA40DzNfKLBkBw38gBkH2CNAnuJjeBruCH\n6Ha7KZ1eGg6KefPmUVRURGJi4uj/oSM02qHwM+BxVX0ndP9U4HJV/cpICx2qWAgFv99PQ0MD9fX1\nh91q6+r4aO9H+P2hvs4iaNpx+DLy8GdMJpCZhyZlwEhOkFFFPG24Whtwt+0joa0B6TgQXPsAbncC\nU6ZOoaiwkMLCQvLz8ykoKKCgoIC8vDwLjBBVpaGhgV27dvXeqqvZtaua9o5gv4pEF0zP8jEn28uc\nbB+zsn2kjbBh5g1AVYub7U2JbG9OYEdLEp3e4GeXnJRIUVERJaXTKSkpCd+mTp1qLUKCLba6ujrq\n6uqora2lvr6emtoa6urq6OoM7ckWkAmCP9sPk0AnhVoDo5m1SrDrzUGQg8GgcDW7UE/wcxQRJk+Z\nTElxCYWFhRQVFVFQUEBhYSG5ubmOnSA3KqEgIpsI/gsSgTlAbeh+MbBFVY8fnXIjN1ahEAgEaGxs\npK6uboAV/97DTnIRdyKakoUvKZNAShaaMoFAyoTgLiH3GGwx+L242g/g6jqEdB3C1dVCgqcF6WpB\n/b11JiQkMGXqVIoKC8NB0ffLeixupfbsRjh85V9FTU0NnV29B4GzkoVpaV7y033kpwcozvRRmukn\nKcoZGlCoa3NT1eJmT7ub3e1u9nQksr+zd57EBDeFhYUfC4uCgoJjLiz8fj979uw5bOVfV1fHrtpd\nNB88/PwPV4YLf7ofzVTIBM1WyGZorYDRokAbSJNAC9AK7nY3tIL6etexySnJ5OfnhwOj51ZcXBz1\nY06jFQpHPUFNVWuO8tyfACuAhoHCQ4Jx+UNgOdABXKGq6wYreLRDob29/bAvX21tLTU1tdTvrsfT\np+eIuBP6rPgnoClZ4QDQxLSRbf1Hiyri7QgHhXS14Oo6RIKnFelsQQO9gZGUnExBfgHFxUUUFRWF\nt3AKCwvj5uBad3c31dXVVFRUhG47qayoCG/5A0xMEfLTPExL95Of7ic/PUB+up/MpNjqYdLpozck\n2t3sbnexuyOJxo7g+gcgKTGR0tISZs6azcyZM5k5cyYzZswgIyPDwcoj19TURGVlJVVVVVRWVlJR\nWcGu6l14vd7wPK5kF4GMAIHMYHdRzQgGABlAPDR6e1oVrSBtEvzZKrjb3QTaAr0fJpA3JY9ZM2Yx\nffp0ZsyYwfTp00c1+Ed199EwCzgLaAN+cYRQWE7wDOnlwKnAD1X11MFed7ih4PP5WLduHdXV1dTW\n1lJbW8uumloONTf1LQpJycKb3GdrP2VCcMWfFKMr/uFSRTw9gXEo3MpI7G5Bu1rCu6MAsrMnhsOi\nqKiI0tJSFi9e7OhWaldXFxs2bKCiooLKykp27thOXV09gVDdKQlCYYaPogwvxRl+CjKCAZCeGFsr\n/6Hq9sPedjf17W5qW93UtrmpaU+itbv375oyOZeZs+aEg2LRokWOHtT2+/3hz6knAHZW7KTlUEt4\nHleqC3+WH52gkAWapcEVf3wcwx0eP9BOMChaBA6Bu8WNtmg4LBISEyguLmbWzN6wmDNnDllZWUN+\nO8dDIVRECbD6CKHwX8ArqvpM6P52YKmq7j3aaw41FFSVt956i0cfe4y60BnDkpSKPzkLf2iF37vy\nzwKXc5sfrtZ9uFv24s+aSiBzsmN1EPCHWxU9YeEO3dQb3HdbVFTM9ddfx+mnnz7m+0irq6v5t2/f\nRv3uPQDkpEJRuoeiTD9FGX6KMv3kpQY+1iMo2nY2u9nalMi8iV5mZfvH7H1Vodkj1La6qWlNCAZF\nWyIftQsKZGakc/sd3+HUUwfd5hp1ra2t3HrrreGTMcUt6AQlkBWACQRDYAJ9RlRzyAGQBkHzFD7e\nEXBs+YEWkEPBoJBDgrvFTaAzeHA7JSWFu+++m1NOOWVILxsPobAauF9V3wjd/xtwi6oedY0/lFDY\nsWMHDz/8MBs3boTUbLryF+OfMA0SnP4GfpyrdR8Ta15mxfLlrF6zhqbis50NhiPxduFu2UPK7nXQ\n2cyiRYtYuXIls2fPHpO3f+WVV7j3nntIppsvzWlj3kRfTGz972x284OteSxbvoI/r1nNTfMaxjQY\nBtLth5pWNz/bnkFdm4urrrqaL37xi2MW4vv27ePmb9xMXX0d/kV+dHJo6z/WGtwHIP29dFacv4LV\nL6ym/ZR254NhIN1AMyRsTEBahVtvuZVly5ZF/PRIQ8HJI4sDfTUGXLpF5FoRKReR8sbGxojf4IEH\nHggGAuBPTAtu+bYfBL93kGeOPXfLXlYsX87Kr93AiuXn4245aoPJGX4vro6DuLoO4U8MHhTbuHEj\nDzzwwJi8fXt7O9/5znfo6u4mO8nPnnY3H3W4CDifCWxtSmTZ8hV89Wsr+fTyFWxtcrZLoioc6HJR\n05rAhCQ/qvDUU0+xYcOGMavh+9//PrU1tWiSBo8athEcICfGSIOw4vwVrPzaSlacvwJpiLXUAgJA\nR7DVEEgKEPAHuPfee/noo49G/a2c7LpQz+GD6hUAewaaUVWfAJ6AYEsh0je49957ef/999m0aRMb\nNm6itmZt8BciaHoOvvQ8/Jl5aEo2gZRMcCcN928ZMX/WVFavWQMoq9e8gL/4bMdqCRbkwdXVinQ1\n425tIKF9H9Le2/W1qLiEE/6/f2DhwoWcfPLJY1JSeno6Dz74IG+99Rbl77/Hbyp3QWUq6YnCvOxu\nFkzyMXeilylpARLHeHNn3kQOmkgaAAATH0lEQVQvP1izGgX+smY1N80b27VfQKGh00XFoQQ+PJjA\nlqZkDoZ6aU6ZnMdnPnMyJ598MgsXLhyzmq677jpmzpzJ+g3r2b5tO/7twZaTZAv+HD+ao5BDcNeR\ng+thzVNWv7AagNUvrEZPiYGtDB/QDNIoyH7BdcCFhrouT54ymZOWnURZWRl5eXmj/tZO7j76DHAD\nvQeaH1bVQXeSjaT3UWtra+iEsg/ZuGkTW7ZswdtnqFxJSsOfnEkgKZNASiaaHOxhFEjJhITUqB9o\nHtNjCqrg68TV1RrsmdTdEpz2tOLubkE9vT12EpOSmD9/PosWLmThwoUsWLCAzMzM6NYXgYMHD/LB\nBx9QXl7O2vL3+WhfAwAugclpMC3Uy2haWiD4M91PahQ3g8bimII3AB91uMK9kva0u9nTkcDedlf4\npLgJWZksXlLGkiVLKCsrY9q0aVGpZSi6urrCJwRu2LCBTR9uwtMdXPYkUdAMJZARCPcs6ulmylg1\nuJw4phCg90Bz/55JHYHwbEXFRSw+aTGLFi1i0aJFww4Cx48piMgzwFKC2wL7gDsIfcSq+nioS+oq\nYBnBxuWXBjueAKPbJbVn+In6+nr27NnD7t272b17N3X1uzl4YP9hA2FJQiKanIUvKSPYFTU5K3iA\nOi17TAJjWHpW/B3NwQPG3S2hcxjakO4W1Ne7NSsiTDouh8KCfPLzg7dp06ZRUFDA9OnT46I//O7d\nu9m6dSs1NTXh8xF2796Dz9+7gj4uFaalesMhkZ8eoChz5CeljTaPP3j+Qu/K38WeziT2dfR2DBMR\npkzOo6R0OsXFxZSUlDBr1ixmzJgR8+ec+Hw+duzYwbZt28LnJNTU1tCwr+Gw5c6V6iKQfoQuqbH9\nJwYpwWMBLYev+F3tLrRNg8EQkpGZQVFhb5fw0tJSFi5cyIQJRxuLI3KOh0K0jNXJa93d3Xz00UeH\nhcWePXuoq6//+MlriSn4U7Lxp2YTSJ1IIHUimpqNJo5RWKgi3k6kswlXZzOuzibcnc24u5rDvYWg\n9+S1woICpk2bdtjKf8qUKXEzhstQ+Hw+9uzZw65du6ipqQkFRjW1NbV09TkPJS8NikM9mEoy/RRl\n+piUrGPy8bV5hV2twS6mu1rd1LYnsadNwsdKEtxuCgrywyv/ngAoLCw85j4zj8dz2MlrPWFRW1t7\nWBdWcQlkgT/LH+zC2tOLKQ3ndkV5CfcW6uk95G5xE+juXfMnJiaSX9B78lrPyaOFhYWjtvI/EguF\nKOo527m2tjZ8lmx1dTVV1dV0tLeH5wuGxQT8qRMJpGYTyMgjkJ474mEuXO2NuNoacHU24+5sOqyr\nKEBaejrTS0spLS0Nn/1aVFR0zJ61PBw9n2HPyW47d+5k5/Zt1O/pPcCfmSwUp3sozvRTnOljdraP\nnJSRLS9tXmFbUwI1rW5qQgHQ9+zlnOMmMWv2HGbPDp6QVlpayrRp0+KipRZtra2twVEFamuprq6m\nurqanRU72d/Ye+0KSRQ0q1+X12xgNA8XBvh4l9FWN4H23pV/SmoK00uD5xX0LIdOjxxgoeAAVeXg\nwYOHDalQVRUMi/a24HURJDkdz4RC/BOL8WdNi+y8iIAfd8se3E01JDXXop7ggHjpGZlMn17K9D4r\n/5KSEiZNmmQXIB+mjo4OKisr2bFjRzAsdmynuroary+4C2rGBD+fyOvmlMkejoswINq9wtrGRN7Z\nl8Tmg4n4Nbjrp7Agn9lz5jJr1ixmzpzJrFmzbOTUYWhrawuHRFVVFVVVVVRUVoQHxEOAHAhMCaDT\nQrufhrp4eIID4rEH3PvcaHfPWGNuioqLmDkjGOA9ITB58uSYWwYtFGJIT1isW7eO119/nbffeYfu\nri4kIRFvVgG+SaX4J5Ue3oJQxX2wmoSD1SS21KM+L8kpKZz2iU9w5plnsnjxYlv5jxGv10tNTQ3v\nvvsuf//bS+ysCF6Va3Z2MCBOn+oho9+5Et4AvLsviXf3JbHxYCL+QLAX0DnnfoozzjiDmTNnkpqa\n6sSfMy70LHNVVVVs2LCBN958g6rKKgAkQ/BP9aNTFXI58rGJNpA9gmuvC/YDAcjMyuT0007nlFNO\nYdasWXE1/pSFQgzzeDysW7eON954g9def53mpiZ8x82ge/qZ4EqAgI/kqtdJOFBJ9sSJnHXmmXzy\nk59k8eLFJCU5123WBNXV1fHyyy/z97+9RFX1Lialws2LWijODLYmDnULD27MpPKQm7zc4zj7nE9x\n9tlnM2/ePAtxB+3bt4933nmHN958g7Vr1+Lz+pBJgu8UX7D10CMA8qHg2h5Mi+KSYs785Jmcdtpp\nzJ8/P25HGrZQiBOBQICnn36aJ598kkDmZLpKzyS1+nWkdR/XXHMNl156qR0HiGFbtmzh3//t27Qd\nauKG41vISQnwnxsm0BZI4tbbvs3SpUvt84tBnZ2dvPbaa/zghz+go6sD/4l+tEShDRLeS0APKp/5\nzGe47LLLmDp1qtPljgoLhTjzyiuvcOedd+L3+3G73dxxxx0sXbrU6bJMBPbv388t3/omu6oqSU1Q\nEtIncv93v8fcuXOdLs0MoqGhgXvuuYcPPviAwJIA7s1u0hPTufWWWznrrLOcLm9UxcMwF6aPpUuX\nhge4OvXUUy0Q4khOTg7fuuVWvAFo8QhXXX2NBUKcyMvL46GHHqKwqBDXWhfapdx3733HXCAMhYVC\nDCktLT3sp4kffQcEPPPMMx2sxAyVy+Xi4osuDk8vWrTI4YqcZaEQQ3qu6xqL13c1R9f3ALJ1K40/\nc+bMAYKf3XjvDGChEEPG+5fRGKdMmjTJ6RJihoVCDLJwMGZs9XQztWXPQiEmxVuPMGOOFbbsWSgY\nY0w4DKylYKFgjDGmDwsFY8y4Zy2EXhYKxhhjwiwUjDHGhFkoGGOMCbNQMMYYE2ahYIwxJsxCwRhj\nTFhUQ0FElonIdhGpEJFbB/j9FSLSKCLrQ7ero1mPMcaYo4vaxUVFxA08AvwfoB54X0SeU9Ut/Wb9\ntareEK06jDHGRC6aLYVTgApVrVJVD/AscEEU388YY8wIRTMU8oG6PvfrQ4/1d6GIbBSR34pIYRTr\nMcYYM4hohsJA5433H4LweaBEVRcBLwE/H/CFRK4VkXIRKW9sbBzlMo0xxvSIZijUA323/AuAPX1n\nUNUDqtoduvsksGSgF1LVJ1S1TFXLcnNzo1KsMcaY6IbC+8AsESkVkSTgEuC5vjOIyNQ+dz8LbI1i\nPcYYYwYRtd5HquoTkRuAvwBu4CequllE7gLKVfU5YKWIfBbwAQeBK6JVjzHGmMFFLRQAVHUNsKbf\nY7f3mb4NuC2aNRhjjImcndFsjDEmzELBGGNMmIWCMcaYMAsFY4wxYRYKxhhjwiwUjDHGhFkoGGOM\nCbNQMMYYE2ahYIwxJsxCwRhjTJiFgjHGmDALBWOMMWEWCsYYY8IsFIwxxoRZKBhjjAmzUDDGGBNm\noWCMMSbMQsEYY0JU1ekSHGehEINExOkSjBmXbNmzUIhJtrVizNiyZa6XhUIMsq2V+Nbd3e10CWaY\nbNmLciiIyDIR2S4iFSJy6wC/TxaRX4d+/66IlESznnhhWy3xrbW11ekSzDDZshfFUBARN/AIcD4w\nH/iCiMzvN9tVQJOqzgQeAr4brXriiW2txJ++KxMLhfhly150WwqnABWqWqWqHuBZ4IJ+81wA/Dw0\n/VvgXLFPxbZW4pDH4wlPt7W1OViJGQlb9qIbCvlAXZ/79aHHBpxHVX3AIeC4KNYUFywX409XV9eA\n0yY+2DLXK5qhMNB/uX8MRzIPInKtiJSLSHljY+OoFBfLbGsl/ni93gGnTXyxZS+6oVAPFPa5XwDs\nOdI8IpIATAAO9n8hVX1CVctUtSw3NzdK5RozfG63e8BpE1+sxRDdUHgfmCUipSKSBFwCPNdvnueA\ny0PTFwF/V4tqE4cSExPD0wkJCQ5WYobDVju9ovbtVVWfiNwA/AVwAz9R1c0ichdQrqrPAT8Gfiki\nFQRbCJdEqx5joiktLS08nZmZ6WAlZiSspRDFUABQ1TXAmn6P3d5nugu4OJo1GDMWXK7eRndGRoaD\nlZiRsBaDndFszKjLyspyugQzTNZSsFCISfbFjG99dyWZ+GItBQuFmJKcnHzYTxOf+u5KMvHBNsR6\nWTeJGLJixQrq6upYsWKF06WYYfjsZz/L9u3bnC7DDENqaioAn/70px2uxHkSb82lsrIyLS8vd7oM\nYwakqrbVGafq6+uZPHnyYd2LjyUislZVywabz1oKxowiC4T4VVBQ4HQJMcF2fhpjjAmzUDDGGBNm\noWCMMSbMQsEYY0yYhYIxxpgwCwVjjDFhFgrGGGPC4u7kNRFpBGqcriOKcoD9Thdhhs0+v/h1rH92\nxao66FXK4i4UjnUiUh7JWYcmNtnnF7/sswuy3UfGGGPCLBSMMcaEWSjEniecLsCMiH1+8cs+O+yY\ngjHGmD6spWCMMSbMQiGGiMgrIlIWml4jItlO12QOJyJtTtdghkZESkTkwwEev0tEPjXIc78jIt+I\nXnWxx66nEKNUdbnTNRhzLFPV252uIRZZS2GEQlsh20TkKRH5UESeFpFPicibIrJTRE4RkXQR+YmI\nvC8iH4jIBaHnporIsyKyUUR+DaT2ed1dIpLTfytHRL4hIt8JTb8iIg+JyGsislVEThaR34fe9+6x\n/l+MJxL0/dBnvklE/jn0+KMi8tnQ9B9E5Ceh6avsM3GUW0SeFJHNIvLX0LL3MxG5CEBEloeW4zdE\n5GERWd3nufNDy1qViKx0qP4xYy2F0TETuBi4Fngf+Bfgk8BngW8DW4C/q+qVoV1C74nIS8CXgQ5V\nXSQii4B1w3hvj6qeJSI3An8ClgAHgUoReUhVD4z0jzMD+jxwInACwTNh3xeR14DXgDOB54B8YGpo\n/k8CzzpQpwmaBXxBVa8Rkd8AF/b8QkRSgP8CzlLVahF5pt9z5wJnA5nAdhF5TFW9Y1X4WLOWwuio\nVtVNqhoANgN/02C3rk1ACXAecKuIrAdeAVKAIuAs4L8BVHUjsHEY7/1c6OcmYLOq7lXVbqAKKBz2\nX2QG80ngGVX1q+o+4FXgZOB14EwRmU9wY2CfiEwFTgPecqxaU62q60PTawkulz3mAlWqWh263z8U\n/ldVu1V1P9AATI5qpQ6zlsLo6O4zHehzP0Dwf+wHLlTV7X2fFLqe72B9gn0cHt4pR3jvvu/b971N\ndAx4MWZV3S0iE4FlBFsNk4B/AtpUtXUM6zOH67ts+Omzq5YjfJZHee4xvVxZS2Fs/AX4moRSQERO\nCj3+GnBp6LHjgUUDPHcfkCcix4lIMrBiDOo1g3sN+GcRcYtILsFW33uh370N3BSa53XgG6GfJjZt\nA6aLSEno/j87V4rzjunEiyH/AfwA2BgKhl0EV+6PAT8VkY3AenpXKmGq6hWRu4B3gWqCX2DjvD8Q\n3CW0gWBr71uq+lHod68D56lqhYjUEGwtWCjEKFXtFJGvAH8Wkf0MsByOJ3ZGszFm3BORDFVtC220\nPQLsVNWHnK7LCbb7yBhj4JpQR5DNwASCvZHGJWspGGOMCbOWgjHGmDALBWOMMWEWCsYYY8IsFIwx\nxoRZKJhjhogMOoyEiNwkImlRruNEETnqKLcicoWIrBrl9x311zTjj4WCOWao6ukRzHYTMKRQEBH3\nEEs5EbChz01cslAwx4yeC+CIyNLQUMe/DQ2H/HRoqOuVwDTgZRF5OTTveSLytoisE5H/EZGM0OO7\nROR2EXkDuFhEZojIn0VkrYi8LiJzQ/NdHBo+e0NoCPMk4C6CQ2Cs7xlSe5C6c0Xkd6Gh1d8XkTNE\nxBWqIbvPfBUiMnmg+Uf9n2nGLRvmwhyrTgIWAHuAN4EzVPVhEfk6cLaq7heRHODfgU+paruI3AJ8\nneBKHaBLVT8JICJ/A65T1Z0icirwKHAOcDvw6dBAeNmq6hGR24EyVb0hwlp/CDykqm+ISBHwF1Wd\nJyJ/Av6R4FAopwK7VHWfiPyq//zAvBH+v4wBLBTMses9Va0HCJ2pWgK80W+eTwDzgTdDYxUmERzM\nrsevQ8/PAE4H/ic0H0By6OebwM9CY/T/fpi1forghVx67meJSGbo/W8Hfgpc0lPPUeY3ZsQsFMyx\nKpLhjgV4UVW/cITXaA/9dAHNqnpi/xlU9brQVvxngPUi8rF5IuACTlPVzsOKE3kbmBkahfVzwN2D\nzD+MtzbmcHZMwYw3rQSvoAXwDnCGiMwEEJE0EZnd/wmq2gJUi8jFoflERE4ITc9Q1XdD1/vdT/DC\nRn3fIxJ/BcK7mnqCJXShpj8ADwJb+1xFb8D5jRkNFgpmvHkCeEFEXlbVRuAK4JnQ8OXvELwK10Au\nBa4SkQ0EB027IPT49yV4jeYPCV4/YQPwMsHdOxEdaAZWAmUSvFb3FuC6Pr/7NfCv9O46Gmx+Y0bE\nBsQzxhgTZi0FY4wxYXag2ZgoEpEvATf2e/hNVf2qE/UYMxjbfWSMMSbMdh8ZY4wJs1AwxhgTZqFg\njDEmzELBGGNMmIWCMcaYsP8fvAZEerH5aLAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f49b1c710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train.loc[train['bathrooms'] > 3, 'bathrooms'] = 3\n",
    "test.loc[test['bathrooms'] > 3, 'bathrooms'] = 3\n",
    "sns.violinplot(x='interest_level', y='bathrooms', data=train)\n",
    "print(train['bathrooms'].value_counts().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面浴室大部分都是1个，但是最大值是10，不太正常统一一下，超过3个也算三个\n",
    "在不同的感兴趣程度上，好像浴室的分不都差不多。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 price"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAFBCAYAAADnkR6vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG6RJREFUeJzt3X+w3XV95/HniwAaazUg0YUEGqpZ\nKk63oHcwls7WWpdEuzXZXV1hakWXltXqVsddttDd0VnrH3aYqZZpbUurLXasgJQi7WrTLNC19Qdy\nESQipaSKNYlTUAhqzSLE9/5xPhcPl5vcc5Oce+7N5/mYOXO/3/f38z3fzyffnOR1vz/ON1WFJEmS\n+nHUpDsgSZKkxWUAlCRJ6owBUJIkqTMGQEmSpM4YACVJkjpjAJQkSeqMAVCSJKkzBkBJkqTOGAAl\nSZI6c/SkO7DUnXDCCbVu3bpJd0OSJGlet95669eravV87QyA81i3bh3T09OT7oYkSdK8knxllHae\nApYkSeqMAVCSJKkzBkBJkqTOGAAlSZI6YwCUJEnqjAFQkiSpMwZASZKkzvg9gEvAdbft4tKtd7N7\nz15OWrWSizaexpYz10y6W5Ik6QhlAJyw627bxSXXbmfvI/sA2LVnL5dcux3AEChJksbCU8ATdunW\nux8LfzP2PrKPS7fePaEeSZKkI50BcMJ279m7oLokSdKhMgBO2EmrVi6oLkmSdKgMgBN20cbTWHnM\nisfVVh6zgos2njahHkmSpCOdN4FM2MyNHt4FLEmSFosBcAnYcuYaA58kSVo0ngKWJEnqjAFQkiSp\nMwZASZKkzhgAJUmSOmMAlCRJ6owBUJIkqTMGQEmSpM4YACVJkjpjAJQkSeqMAVCSJKkzBkBJkqTO\nGAAlSZI6YwCUJEnqjAFQkiSpMwZASZKkzhgAJUmSOmMAlCRJ6owBUJIkqTMGQEmSpM6MPQAmWZHk\ntiR/0eZPTXJzknuSXJXk2FZ/Upvf0ZavG3qPS1r97iQbh+qbWm1HkouH6gvehiRJUi8W4wjgW4C7\nhuZ/HXhPVa0HHgQuaPULgAer6jnAe1o7kpwOnAs8D9gEvK+FyhXAbwMvA04HzmttF7wNSZKknow1\nACZZC/wM8AdtPsBLgGtakyuALW16c5unLf/p1n4zcGVVPVxVXwZ2AGe1146q+lJVfRe4Eth8kNuQ\nJEnqxriPAL4X+O/A99r8M4A9VfVom98JrGnTa4CvArTlD7X2j9VnrbO/+sFs43GSXJhkOsn0/fff\nv/BRS5IkLWFjC4BJ/i1wX1XdOlyeo2nNs+xw1efb/vcLVZdX1VRVTa1evXqOVSRJkpavo8f43mcD\nr0jycuDJwNMYHBFcleTodgRuLbC7td8JnAzsTHI08HTggaH6jOF15qp//SC2IUmS1I2xHQGsqkuq\nam1VrWNwE8eNVfVzwE3AK1uz84GPtunr2zxt+Y1VVa1+bruD91RgPfBZ4BZgfbvj99i2jevbOgvd\nhiRJUjfGeQRwf34FuDLJu4DbgPe3+vuBP06yg8FRuXMBqurOJFcDXwQeBd5UVfsAkrwZ2AqsAD5Q\nVXcezDYkSZJ6Eg+AHdjU1FRNT09PuhuSJEnzSnJrVU3N184ngUiSJHXGAChJktQZA6AkSVJnDICS\nJEmdMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqS\nJHXGAChJktQZA6AkSVJnDICSJEmdMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS\n1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJktQZA6AkSVJnDICSJEmdMQBKkiR1xgAoSZLUGQOgJElS\nZwyAkiRJnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJktQZA6AkSVJnDICSJEmd\nMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS1JmxBcAkT07y2SSfT3Jnkv/V6qcm\nuTnJPUmuSnJsqz+pze9oy9cNvdclrX53ko1D9U2ttiPJxUP1BW9DkiSpF+M8Avgw8JKq+jHgDGBT\nkg3ArwPvqar1wIPABa39BcCDVfUc4D2tHUlOB84FngdsAt6XZEWSFcBvAy8DTgfOa21Z6DYkSZJ6\nMrYAWAPfbrPHtFcBLwGuafUrgC1tenObpy3/6SRp9Sur6uGq+jKwAzirvXZU1Zeq6rvAlcDmts5C\ntyFJktSNsV4D2I7U3Q7cB2wD/gHYU1WPtiY7gTVteg3wVYC2/CHgGcP1Wevsr/6Mg9jG7H5fmGQ6\nyfT9999/cIOXJElaosYaAKtqX1WdAaxlcMTuuXM1az/nOhJXh7F+oG08vlB1eVVNVdXU6tWr51hF\nkiRp+VqUu4Crag/w18AGYFWSo9uitcDuNr0TOBmgLX868MBwfdY6+6t//SC2IUmS1I1x3gW8Osmq\nNr0SeClwF3AT8MrW7Hzgo236+jZPW35jVVWrn9vu4D0VWA98FrgFWN/u+D2WwY0i17d1FroNSZKk\nbhw9f5ODdiJwRbtb9yjg6qr6iyRfBK5M8i7gNuD9rf37gT9OsoPBUblzAarqziRXA18EHgXeVFX7\nAJK8GdgKrAA+UFV3tvf6lYVsQ5IkqSfxANiBTU1N1fT09KS7IUmSNK8kt1bV1HztfBKIJElSZwyA\nkiRJnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJktQZA6AkSVJnDICSJEmdMQBK\nkiR1xgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJ\nktQZA6AkSVJnDICSJEmdMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJnRk5ACb5oSQvbdMrk/zg+Lol\nSZKkcRkpACb5ReAa4PdaaS1w3bg6JUmSpPEZ9Qjgm4CzgW8CVNU9wDPH1SlJkiSNz6gB8OGq+u7M\nTJKjgRpPlyRJkjROowbA/5vkV4GVSf4N8BHgz8fXLUmSJI3LqAHwYuB+YDvwn4GPAf9zXJ2SJEnS\n+Bw9YruVwAeq6vcBkqxote+Mq2OSJEkaj1GPAN7AIPDNWAn8n8PfHUmSJI3bqAHwyVX17ZmZNv2U\n8XRJkiRJ4zRqAPznJM+fmUnyAmDveLokSZKkcRr1GsC3Ah9JsrvNnwi8ejxdkiRJ0jiNFACr6pYk\nPwKcBgT4u6p6ZKw9kyRJ0lgcMAAmeUlV3Zjk389atD4JVXXtGPsmSZKkMZjvCOBPAjcCPzvHsgIM\ngJIkScvMAQNgVb0jyVHAx6vq6kXqkyRJksZo3ruAq+p7wJsXoS+SJElaBKN+Dcy2JP8tyclJjp95\njbVnkiRJGotRvwbmPzG45u+XZtV/+PB2R5IkSeM2agA8nUH4+wkGQfBvgN8dV6ckSZI0PqMGwCuA\nbwKXtfnzWu0/jqNTkiRJGp9RrwE8rap+oapuaq8LGXwp9H616wVvSnJXkjuTvKXVj0+yLck97edx\nrZ4klyXZkeSOWY+eO7+1vyfJ+UP1FyTZ3ta5LEkOdhuSJEm9GDUA3pZkw8xMkhcCn5xnnUeB/1pV\nzwU2AG9KcjpwMXBDVa0HbmjzAC8D1rfXhcDvtG0dD7wDeCFwFvCOmUDX2lw4tN6mVl/QNiRJknoy\nagB8IfCpJPcmuRf4NPCT7ejbHXOtUFVfq6rPtelvAXcBa4DNDE4f035uadObgQ/WwGeAVUlOBDYC\n26rqgap6ENgGbGrLnlZVn66qAj44670Wsg1JkqRujHoN4Kb5m+xfknXAmcDNwLOq6mswCIlJntma\nrQG+OrTazlY7UH3nHHUOYhtfm9XfCxkcIeSUU05Z2GAlSZKWuJECYFV95WA3kOSpwJ8Cb62qb7bL\n9OZsOtemD6J+wO6Msk5VXQ5cDjA1NTXfe0qSJC0ro54CPihJjmEQ/j5UVTPPDf6nmdOu7ed9rb4T\nOHlo9bXA7nnqa+eoH8w2JEmSujG2ANjuyH0/cFdV/cbQouuBmTt5zwc+OlR/bbtTdwPwUDuNuxU4\nJ8lx7eaPc4Ctbdm3kmxo23rtrPdayDYkSZK6Meo1gAfjbODnge1Jbm+1XwXeDVyd5ALgH4FXtWUf\nA14O7AC+A7weoKoeSPJrwC2t3Tur6oE2/Ubgj4CVwMfbi4VuQ5IkqScZ3ECr/Zmamqrp6elJd0OS\nJGleSW6tqqn52o31GkBJkiQtPQZASZKkzhgAJUmSOmMAlCRJ6owBUJIkqTMGQEmSpM4YACVJkjpj\nAJQkSeqMAVCSJKkzBkBJkqTOGAAlSZI6YwCUJEnqjAFQkiSpMwZASZKkzhgAJUmSOmMAlCRJ6owB\nUJIkqTMGQEmSpM4YACVJkjpjAJQkSeqMAVCSJKkzBkBJkqTOGAAlSZI6YwCUJEnqjAFQkiSpMwZA\nSZKkzhgAJUmSOmMAlCRJ6owBUJIkqTMGQEmSpM4YACVJkjpjAJQkSeqMAVCSJKkzBkBJkqTOGAAl\nSZI6YwCUJEnqjAFQkiSpMwZASZKkzhw96Q4IrrttF5duvZvde/Zy0qqVXLTxNLacuWbS3ZIkSUco\nA+CEXXfbLi65djt7H9kHwK49e7nk2u0AhkBJkjQWngKesEu33v1Y+Jux95F9XLr17gn1SJIkHekM\ngBO2e8/eBdUlSZIOlQFwwk5atXJBdUmSpENlAJywizae9oSdcFSrS5IkjYMBcMKmv/IA35tV+16r\nS5IkjYMBcMI+fPNXF1SXJEk6VGMLgEk+kOS+JF8Yqh2fZFuSe9rP41o9SS5LsiPJHUmeP7TO+a39\nPUnOH6q/IMn2ts5lSXKw25ikfVULqkuSJB2qcR4B/CNg06zaxcANVbUeuKHNA7wMWN9eFwK/A4Mw\nB7wDeCFwFvCOmUDX2lw4tN6mg9nGpK0Y5NaR65IkSYdqbAGwqj4BzL6QbTNwRZu+AtgyVP9gDXwG\nWJXkRGAjsK2qHqiqB4FtwKa27GlV9emqKuCDs95rIduYqPNeePKC6pIkSYdqsa8BfFZVfQ2g/Xxm\nq68Bhi9629lqB6rvnKN+MNuYqHdt+VFes+GUx474rUh4zYZTeNeWH51wzyRJ0pFqqTwKbq7znXUQ\n9YPZxhMbJhcyOE3MKaecMs/bHrp3bflRA58kSVo0i30E8J9mTru2n/e1+k5g+JznWmD3PPW1c9QP\nZhtPUFWXV9VUVU2tXr16QQOUJEla6hY7AF4PzNzJez7w0aH6a9uduhuAh9rp263AOUmOazd/nANs\nbcu+lWRDu/v3tbPeayHbkCRJ6srYTgEn+TDwYuCEJDsZ3M37buDqJBcA/wi8qjX/GPByYAfwHeD1\nAFX1QJJfA25p7d5ZVTM3lryRwZ3GK4GPtxcL3YYkSVJvUn7f3AFNTU3V9PT0pLshSZI0ryS3VtXU\nfO18EogkSVJnDICSJEmdMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS1BkDoCRJ\nUmcMgJIkSZ0xAEqSJHXGAChJktQZA6AkSVJnDICSJEmdMQBKkiR1xgAoSZLUGQOgJElSZwyAkiRJ\nnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJktQZA6AkSVJnDICSJEmdMQBKkiR1\nxgAoSZLUGQOgJElSZwyAkiRJnTEASpIkdcYAKEmS1BkDoCRJUmcMgJIkSZ0xAEqSJHXGAChJktQZ\nA6AkSVJnDICSJEmdMQBKkiR15uhJd0Bw3W27uHTr3ezes5eTVq3koo2nseXMNZPuliRJOkIZACfs\nutt2ccm129n7yD4Adu3ZyyXXbgcwBEqStIwt5QM8ngKesEu33v1Y+Jux95F9XLr17gn1SJIkHaqZ\nAzy79uyl+P4Bnutu2zXprgEGwInbtWfvguqSJGnpW+oHeAyAkiRJh9nu/RzI2V99sRkAJUmSDrOT\nVq1cUH2xGQAlSZIOs4s2nsbKY1Y8rrbymBVctPG0CfXo8bq7CzjJJuA3gRXAH1TVuyfcJUkdWXfx\n/35C7d53/8wEeiJpnLacuYa3XnX742p7H9nnXcCTkGQF8NvAy4DTgfOSnD7ZXknqxVzh70B1ScvX\nUv+8dxUAgbOAHVX1par6LnAlsHnCfZIkSVpUvQXANcBXh+Z3tpokSVI3eguAmaNWT2iUXJhkOsn0\n/fffvwjdkiRJWjy9BcCdwMlD82uB3bMbVdXlVTVVVVOrV69etM5JkiQtht4C4C3A+iSnJjkWOBe4\nfpIdOvvZxy+oLmn5Wv/MH1hQXdLy9d5Xn7Gg+mLrKgBW1aPAm4GtwF3A1VV15yT79KFffNETwt7Z\nzz6eD/3iiybUI0njsu1tL35C2Fv/zB9g29tePJkOSRqbLWeu4b2vPoM1q1YSYM2qlbz31Wcsma+B\nSdUTLoHTkKmpqZqenp50NyRJkuaV5NaqmpqvXVdHACVJkmQAlCRJ6o4BUJIkqTMGQEmSpM4YACVJ\nkjpjAJQkSeqMAVCSJKkzfg/gPJLcD3xlkTZ3AvD1RdrWUuPY++TY++TY++TYF8cPVdW8z7E1AC4h\nSaZH+fLGI5Fjd+y9ceyOvTeOfWmN3VPAkiRJnTEASpIkdcYAuLRcPukOTJBj75Nj75Nj75NjX0K8\nBlCSJKkzHgGUJEnqjAFQkiSpMwbARZBkU5K7k+xIcvEcy5+U5Kq2/OYk64aWXdLqdyfZuJj9PhxG\nGPvbknwxyR1JbkjyQ0PL9iW5vb2uX9yeH7oRxv66JPcPjfEXhpadn+Se9jp/cXt+eIww/vcMjf3v\nk+wZWrZs932SDyS5L8kX9rM8SS5rfy53JHn+0LJlvd9HGPvPtTHfkeRTSX5saNm9Sba3fT69eL0+\nPEYY+4uTPDT09/rtQ8sO+FlZ6kYY+0VD4/5C+3wf35Yt9/1+cpKbktyV5M4kb5mjzdL8zFeVrzG+\ngBXAPwA/DBwLfB44fVabXwJ+t02fC1zVpk9v7Z8EnNreZ8Wkx3SYx/5TwFPa9Btnxt7mvz3pMYx5\n7K8DfmuOdY8HvtR+Htemj5v0mA73+Ge1/y/AB46Qff+vgecDX9jP8pcDHwcCbABuPoL2+3xj//GZ\nMQEvmxl7m78XOGHSYxjj2F8M/MUc9QV9Vpbia76xz2r7s8CNR9B+PxF4fpv+QeDv5/i3fkl+5j0C\nOH5nATuq6ktV9V3gSmDzrDabgSva9DXATydJq19ZVQ9X1ZeBHe39lot5x15VN1XVd9rsZ4C1i9zH\ncRllv+/PRmBbVT1QVQ8C24BNY+rnuCx0/OcBH16Uno1ZVX0CeOAATTYDH6yBzwCrkpzIEbDf5xt7\nVX2qjQ2OrM/7KPt9fw7l34olYYFjP2I+6wBV9bWq+lyb/hZwF7BmVrMl+Zk3AI7fGuCrQ/M7eeJf\njsfaVNWjwEPAM0ZcdylbaP8vYPBb0ownJ5lO8pkkW8bRwTEadez/oZ0SuCbJyQtcdykbeQzttP+p\nwI1D5eW87+ezvz+bI2G/L8Tsz3sBf5Xk1iQXTqhP4/aiJJ9P8vEkz2u1bvZ7kqcwCDh/OlQ+YvZ7\nBpdvnQncPGvRkvzMH71YG+pY5qjN/u6d/bUZZd2lbOT+J3kNMAX85FD5lKraneSHgRuTbK+qfxhD\nP8dhlLH/OfDhqno4yRsYHAV+yYjrLnULGcO5wDVVtW+otpz3/XyO1M/7yJL8FIMA+BND5bPbPn8m\nsC3J37UjS0eKzzF4Ruu3k7wcuA5YT0f7ncHp309W1fDRwiNivyd5KoNg+9aq+ubsxXOsMvHPvEcA\nx28ncPLQ/Fpg9/7aJDkaeDqDw+mjrLuUjdT/JC8F/gfwiqp6eKZeVbvbzy8Bf83gN6vlYt6xV9U3\nhsb7+8ALRl13GVjIGM5l1imhZb7v57O/P5sjYb/PK8m/Av4A2FxV35ipD+3z+4A/Y3ld7jKvqvpm\nVX27TX8MOCbJCXSy35sDfdaX7X5PcgyD8Pehqrp2jiZL8jNvABy/W4D1SU5NciyDD8DsuxqvB2bu\n/nklgwtkq9XPzeAu4VMZ/Lb42UXq9+Ew79iTnAn8HoPwd99Q/bgkT2rTJwBnA19ctJ4fulHGfuLQ\n7CsYXDsCsBU4p/0ZHAec02rLySh/70lyGoOLnz89VFvu+34+1wOvbXcGbgAeqqqvcWTs9wNKcgpw\nLfDzVfX3Q/UfSPKDM9MMxj7nHaXLVZJ/0a7tJslZDP7//QYjflaWuyRPZ3CG56NDtWW/39s+fT9w\nV1X9xn6aLcnPvKeAx6yqHk3yZgY7dQWDOx3vTPJOYLqqrmfwl+ePk+xgcOTv3LbunUmuZvCf36PA\nm2adJlvSRhz7pcBTgY+0fxv/sapeATwX+L0k32PwD+W7q2rZhIARx/7LSV7BYN8+wOCuYKrqgSS/\nxuA/BoB3zjplsuSNOH4YXBB+ZfuFZ8ay3vdJPszgjs8TkuwE3gEcA1BVvwt8jMFdgTuA7wCvb8uW\n/X4fYexvZ3B98/va5/3RqpoCngX8WasdDfxJVf3log/gEIww9lcCb0zyKLAXOLf9vZ/zszKBIRy0\nEcYO8O+Av6qqfx5addnvdwa/oP48sD3J7a32q8ApsLQ/8z4KTpIkqTOeApYkSeqMAVCSJKkzBkBJ\nkqTOGAAlSZI6YwCUJEnqjAFQUneSfGqENm9tj64aZz/OaE+FOFCb1yX5rcO83cP+npKWFwOgpO5U\n1Y+P0OytwIICYJIVC+zKGQy+H0ySFpUBUFJ3kny7/Xxxkr9Ock2Sv0vyofZt/b8MnATclOSm1vac\nJJ9O8rkkH2nP/iTJvUnenuRvgVcleXaSv8zg4fZ/k+RHWrtXJflCks8n+UR76sM7gVcnuT3Jq0fo\n9+okf5rklvY6O8lRrQ+rhtrtSPKsudof9j9MScuSTwKR1LszgecxeAbnJxk8nP6yJG8Dfqqqvt4e\nSfc/gZdW1T8n+RXgbQwCHMD/q6qfAEhyA/CGqronyQuB9wEvYfAUjI1VtSvJqqr6bpK3A1NV9eYR\n+/qbwHuq6m/bY9W2VtVzk3yUwZMW/rBt896q+qckfzK7PYMnrUjqnAFQUu8+W1U7AdqjnNYBfzur\nzQbgdOCT7bFVxzL0/GLgqrb+U4Ef5/uPNgR4Uvv5SeCP2uMd53pg/CheCpw+9N5Pa89SvYpBwPxD\nBo+SvGqe9pI6ZwCU1LuHh6b3Mfe/iwG2VdV5+3mPmeebHgXsqaozZjeoqje0o3M/A9ye5AltRnAU\n8KKq2vu4ziWfBp6TZDWwBXjXPO0PYtOSjiReAyhJc/sWMHO07DPA2UmeA5DkKUn+5ewVquqbwJeT\nvKq1S5Ifa9PPrqqbq+rtwNeBk2dtYxR/BTx2ungmRNbgoe5/BvwGcFdVfeNA7SXJAChJc7sc+HiS\nm6rqfuB1wIeT3MEgEP7Iftb7OeCCJJ8H7gQ2t/qlSbYn+QLwCeDzwE0MTtGOdBMI8MvAVJI7knwR\neMPQsquA1/D907/ztZfUsQx+cZQkSVIvPAIoSZLUGW8CkaQlIMnrgbfMKn+yqt40if5IOrJ5CliS\nJKkzngKWJEnqjAFQkiSpMwZASZKkzhgAJUmSOmMAlCRJ6sz/B/HnQ0rf7tU4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f8f5ce9b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,5))\n",
    "plt.scatter(train.interest_level,train.price)\n",
    "plt.xlabel('interest_level')\n",
    "plt.ylabel('price')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里有几个价格特别的偏高，我觉得它们应该是噪声点，去掉> 1000000的这几个样本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1070000    2\n",
       "4490000    1\n",
       "1150000    1\n",
       "Name: price, dtype: int64"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.loc[train['price']>1000000, 'price'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后通过直方图来看看price的分布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = train[train.price <= 1000000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12.0137903186\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm8AAAEyCAYAAACszjc8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFnNJREFUeJzt3X+snXd9H/D3ZzGhFMriEId5cagD\nsqqmlRqClZgxTQy2xIkqAhJICajxaCZXLEiwdRpJkZYOigTbSrtoNJAWjzARQsaPJorMMiuLVFUq\nJg6E/CCk9kKamKSJM/NrQ2qb7rs/ztdwuD7X9/r6x73fm9dLenSe83m+z3Oe89X3Xr/9POd7T7XW\nAgDAGP7Ocp8AAACLJ7wBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4AAAayZrlP\nYKnOOOOMtnHjxuU+DQCABd17773PttbWHY9jDRveNm7cmD179iz3aQAALKiq/uJ4HcttUwCAgQhv\nAAADEd4AAAYivAEADER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR3gAABjLsd5uuJDfv\nfnxR7d5+4StO8JkAAKudK28AAAMR3gAABiK8AQAMRHgDABiI8AYAMBDhDQBgIMIbAMBAFgxvVXV2\nVd1dVQ9X1UNV9Z5e/+2q+k5V3deXS6f2ubaq9lXVI1V18VR9a6/tq6prpurnVNXuqtpbVZ+rqlOP\n9xsFAFgNFnPl7bkkv9la+8UkW5JcXVXn9m2/11o7ry87k6RvuzzJLyXZmuQPquqUqjolyceSXJLk\n3CRXTB3nI/1Ym5J8N8lVx+n9AQCsKguGt9baU621r/X1HyZ5OMlZR9jlsiS3tNb+qrX27ST7klzQ\nl32ttUdba3+d5JYkl1VVJXlDks/3/W9K8ualviEAgNXsqD7zVlUbk7w6ye5eendV3V9VO6pqba+d\nleSJqd3299p89Zcl+V5r7bk59Vmvv72q9lTVngMHDhzNqQMArAqLDm9V9ZIkX0jy3tbaD5LckORV\nSc5L8lSS3z3UdMbubQn1w4ut3dha29xa27xu3brFnjoAwKqxqC+mr6oXZBLcPtNa+2KStNaentr+\nh0nu6E/3Jzl7avcNSZ7s67PqzyY5rarW9Ktv0+0BAJiymNmmleSTSR5urX10qr5+qtlbkjzY129P\ncnlVvbCqzkmyKclXk9yTZFOfWXpqJpMabm+ttSR3J3lr339bktuO7W0BAKxOi7ny9rokv5bkgaq6\nr9d+K5PZoudlcovzsSS/kSSttYeq6tYk38xkpurVrbW/TZKqeneSO5OckmRHa+2hfrz3Jbmlqn4n\nydczCYsAAMyxYHhrrf1pZn8ubecR9vlQkg/NqO+ctV9r7dFMZqMCAHAEvmEBAGAgwhsAwECENwCA\ngQhvAAADEd4AAAYivAEADER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR3gAABiK8AQAM\nRHgDABiI8AYAMBDhDQBgIMIbAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAg\nwhsAwECENwCAgQhvAAADEd4AAAYivAEADER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR\n3gAABiK8AQAMRHgDABiI8AYAMBDhDQBgIAuGt6o6u6rurqqHq+qhqnpPr59eVbuqam9/XNvrVVXX\nV9W+qrq/qs6fOta23n5vVW2bqr+mqh7o+1xfVXUi3iwAwOgWc+XtuSS/2Vr7xSRbklxdVecmuSbJ\nXa21TUnu6s+T5JIkm/qyPckNySTsJbkuyYVJLkhy3aHA19tsn9pv67G/NQCA1WfB8NZae6q19rW+\n/sMkDyc5K8llSW7qzW5K8ua+flmST7eJryQ5rarWJ7k4ya7W2sHW2neT7EqytW97aWvtz1prLcmn\np44FAMCUo/rMW1VtTPLqJLuTvLy19lQyCXhJzuzNzkryxNRu+3vtSPX9M+qzXn97Ve2pqj0HDhw4\nmlMHAFgVFh3equolSb6Q5L2ttR8cqemMWltC/fBiaze21ja31javW7duoVMGAFh1FhXequoFmQS3\nz7TWvtjLT/dbnumPz/T6/iRnT+2+IcmTC9Q3zKgDADDHYmabVpJPJnm4tfbRqU23Jzk0Y3Rbktum\n6lf2Wadbkny/31a9M8lFVbW2T1S4KMmdfdsPq2pLf60rp44FAMCUNYto87okv5bkgaq6r9d+K8mH\nk9xaVVcleTzJ2/q2nUkuTbIvyY+SvDNJWmsHq+qDSe7p7T7QWjvY19+V5FNJXpTky30BAGCOBcNb\na+1PM/tzaUnyxhntW5Kr5znWjiQ7ZtT3JPnlhc4FAOD5zjcsAAAMRHgDABiI8AYAMBDhDQBgIMIb\nAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4A\nAAYivAEADER4AwAYiPAGADCQNct9AivZzbsfX+5TAAD4Ka68AQAMRHgDABiI8AYAMBDhDQBgIMIb\nAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4A\nAAYivAEADER4AwAYiPAGADAQ4Q0AYCALhreq2lFVz1TVg1O1366q71TVfX25dGrbtVW1r6oeqaqL\np+pbe21fVV0zVT+nqnZX1d6q+lxVnXo83yAAwGqymCtvn0qydUb991pr5/VlZ5JU1blJLk/yS32f\nP6iqU6rqlCQfS3JJknOTXNHbJslH+rE2JflukquO5Q0BAKxmC4a31tqfJDm4yONdluSW1tpftda+\nnWRfkgv6sq+19mhr7a+T3JLksqqqJG9I8vm+/01J3nyU7wEA4HnjWD7z9u6qur/fVl3ba2cleWKq\nzf5em6/+siTfa609N6c+U1Vtr6o9VbXnwIEDx3DqAABjWmp4uyHJq5Kcl+SpJL/b6zWjbVtCfabW\n2o2ttc2ttc3r1q07ujMGAFgF1ixlp9ba04fWq+oPk9zRn+5PcvZU0w1Jnuzrs+rPJjmtqtb0q2/T\n7QEAmGNJV96qav3U07ckOTQT9fYkl1fVC6vqnCSbknw1yT1JNvWZpadmMqnh9tZaS3J3krf2/bcl\nuW0p5wQA8Hyw4JW3qvpsktcnOaOq9ie5Lsnrq+q8TG5xPpbkN5KktfZQVd2a5JtJnktydWvtb/tx\n3p3kziSnJNnRWnuov8T7ktxSVb+T5OtJPnnc3h0AwCqzYHhrrV0xozxvwGqtfSjJh2bUdybZOaP+\naCazUQEAWIBvWAAAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4AAAYivAEADER4AwAYiPAG\nADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR3gAABiK8AQAMRHgDABiI8AYAMBDhDQBgIMIbAMBAhDcA\ngIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4AAAYivAEA\nDER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR3gAABiK8AQAMRHgDABjIguGtqnZU1TNV\n9eBU7fSq2lVVe/vj2l6vqrq+qvZV1f1Vdf7UPtt6+71VtW2q/pqqeqDvc31V1fF+kwAAq8Virrx9\nKsnWObVrktzVWtuU5K7+PEkuSbKpL9uT3JBMwl6S65JcmOSCJNcdCny9zfap/ea+FgAA3YLhrbX2\nJ0kOzilfluSmvn5TkjdP1T/dJr6S5LSqWp/k4iS7WmsHW2vfTbIryda+7aWttT9rrbUkn546FgAA\ncyz1M28vb609lST98cxePyvJE1Pt9vfaker7Z9RnqqrtVbWnqvYcOHBgiacOADCu4z1hYdbn1doS\n6jO11m5srW1urW1et27dEk8RAGBcSw1vT/dbnumPz/T6/iRnT7XbkOTJBeobZtQBAJhhqeHt9iSH\nZoxuS3LbVP3KPut0S5Lv99uqdya5qKrW9okKFyW5s2/7YVVt6bNMr5w6FgAAc6xZqEFVfTbJ65Oc\nUVX7M5k1+uEkt1bVVUkeT/K23nxnkkuT7EvyoyTvTJLW2sGq+mCSe3q7D7TWDk2CeFcmM1pflOTL\nfQEAYIYFw1tr7Yp5Nr1xRtuW5Op5jrMjyY4Z9T1Jfnmh8wAAwDcsAAAMRXgDABiI8AYAMBDhDQBg\nIMIbAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAAD\nEd4AAAYivAEADER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICBCG8AAAMR3gAABiK8AQAMRHgDABiI\n8AYAMBDhDQBgIMIbAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAwEOENAGAgwhsAwECE\nNwCAgQhvAAADEd4AAAZyTOGtqh6rqgeq6r6q2tNrp1fVrqra2x/X9npV1fVVta+q7q+q86eOs623\n31tV247tLQEArF7H48rbP26tndda29yfX5PkrtbapiR39edJckmSTX3ZnuSGZBL2klyX5MIkFyS5\n7lDgAwDgp52I26aXJbmpr9+U5M1T9U+3ia8kOa2q1ie5OMmu1trB1tp3k+xKsvUEnBcAwPCONby1\nJP+jqu6tqu299vLW2lNJ0h/P7PWzkjwxte/+Xpuvfpiq2l5Ve6pqz4EDB47x1AEAxrPmGPd/XWvt\nyao6M8muqvrWEdrWjFo7Qv3wYms3JrkxSTZv3jyzDQDAanZMV95aa0/2x2eSfCmTz6w93W+Hpj8+\n05vvT3L21O4bkjx5hDoAAHMsObxV1Yur6ucOrSe5KMmDSW5PcmjG6LYkt/X125Nc2Wedbkny/X5b\n9c4kF1XV2j5R4aJeAwBgjmO5bfryJF+qqkPHubm19t+r6p4kt1bVVUkeT/K23n5nkkuT7EvyoyTv\nTJLW2sGq+mCSe3q7D7TWDh7DeQEArFpLDm+ttUeT/MqM+v9O8sYZ9Zbk6nmOtSPJjqWeCwDA84Vv\nWAAAGMixzjblKNy8+/FFtXv7ha84wWcCAIzKlTcAgIEIbwAAAxHeAAAGIrwBAAxEeAMAGIjwBgAw\nEOENAGAgwhsAwECENwCAgQhvAAADEd4AAAYivAEADER4AwAYiPAGADAQ4Q0AYCDCGwDAQIQ3AICB\nCG8AAAMR3gAABiK8AQAMRHgDABiI8AYAMBDhDQBgIMIbAMBAhDcAgIEIbwAAAxHeAAAGIrwBAAxE\neAMAGIjwBgAwEOENAGAgwhsAwECENwCAgQhvAAADEd4AAAayZrlPgMPdvPvxRbd9+4WvOIFnAgCs\nNK68AQAMRHgDABjIirltWlVbk/ynJKck+aPW2oeX+ZSGsNhbrG6vAsDqsCKuvFXVKUk+luSSJOcm\nuaKqzl3eswIAWHlWypW3C5Lsa609miRVdUuSy5J8c1nPahVxhQ4AVoeVEt7OSvLE1PP9SS5cpnN5\nXjuama6LIQwCwPG1UsJbzai1wxpVbU+yvT/9P1X1yAk9q+SMJM+e4NdYLWb21TuW4UQGYFwtnr46\nOvpr8fTV4umrxTtSX/388XqRlRLe9ic5e+r5hiRPzm3UWrsxyY0n66Sqak9rbfPJer2R6avF01eL\np6+Ojv5aPH21ePpq8U5WX62ICQtJ7kmyqarOqapTk1ye5PZlPicAgBVnRVx5a609V1XvTnJnJn8q\nZEdr7aFlPi0AgBVnRYS3JGmt7Uyyc7nPY46Tdot2FdBXi6evFk9fHR39tXj6avH01eKdlL6q1g6b\nFwAAwAq1Uj7zBgDAIghvAAADEd5mqKqtVfVIVe2rqmuW+3xOlqo6u6rurqqHq+qhqnpPr59eVbuq\nam9/XNvrVVXX9366v6rOnzrWtt5+b1Vtm6q/pqoe6PtcX1Wz/sbfMKrqlKr6elXd0Z+fU1W7+/v+\nXJ89nap6YX++r2/fOHWMa3v9kaq6eKq+asZhVZ1WVZ+vqm/18fVa42p+VfUv+8/gg1X12ar6GWNr\noqp2VNUzVfXgVO2Ej6X5XmMlm6ev/kP/Oby/qr5UVadNbTuq8bKUMblSzeqrqW3/uqpaVZ3Rny//\nuGqtWaaWTGa7/q8kr0xyapJvJDl3uc/rJL339UnO7+s/l+TPM/mu2X+f5JpevybJR/r6pUm+nMkf\nWd6SZHevn57k0f64tq+v7du+muS1fZ8vJ7lkud/3MfbZv0pyc5I7+vNbk1ze1z+e5F19/V8k+Xhf\nvzzJ5/r6uX2MvTDJOX3snbLaxmGSm5L8875+apLTjKt5++qsJN9O8qKpMfXPjK0f988/SnJ+kgen\naid8LM33Git5maevLkqypq9/ZKqvjnq8HO2YXMnLrL7q9bMz+UsYf5HkjJUyrpa9w1ba0jv3zqnn\n1ya5drnPa5n64rYk/zTJI0nW99r6JI/09U8kuWKq/SN9+xVJPjFV/0SvrU/yran6T7Ubbcnkj0nf\nleQNSe7oP5TPTv1i/PFY6j/8r+3ra3q7mju+DrVbTeMwyUszCSM1p25cze6vQ18XeHofK3ckudjY\n+qk+2pifDiQnfCzN9xorfZnbV3O2vSXJZ2aNg4XGy1J+3y13Xyylr5J8PsmvJHksPwlvyz6u3DY9\n3KzvWT1rmc5l2fTL3K9OsjvJy1trTyVJfzyzN5uvr45U3z+jPqrfT/Jvkvy//vxlSb7XWnuuP59+\nfz/uk779+7390fbhiF6Z5ECS/1KTW8x/VFUvjnE1U2vtO0n+Y5LHkzyVyVi5N8bWkZyMsTTfa4zs\n1zO5CpQcfV8t5ffdUKrqTUm+01r7xpxNyz6uhLfDLep7VlezqnpJki8keW9r7QdHajqj1pZQH05V\n/WqSZ1pr906XZzRtC2xb9X2Vyf+8z09yQ2vt1Un+bya3B+bzfO6r9M+8XJbJrau/n+TFSS6Z0dTY\nWpi+mUdVvT/Jc0k+c6g0o9lS+2r4fqyqn03y/iT/dtbmGbWTOq6Et8Mt6ntWV6uqekEmwe0zrbUv\n9vLTVbW+b1+f5Jlen6+vjlTfMKM+otcleVNVPZbklkxunf5+ktOq6tAfv55+fz/uk7797yY5mKPv\nwxHtT7K/tba7P/98JmHOuJrtnyT5dmvtQGvtb5J8Mck/iLF1JCdjLM33GsPpH6T/1STvaP1+XY6+\nr57N0Y/Jkbwqk/9AfaP/nt+Q5GtV9feyAsaV8Ha45+33rPbZL59M8nBr7aNTm25Psq2vb8vks3CH\n6lf2mTdbkny/X/a9M8lFVbW2X0W4KJPPQjyV5IdVtaW/1pVTxxpKa+3a1tqG1trGTMbI/2ytvSPJ\n3Une2pvN7atDffjW3r71+uV9dtY5STZl8sHWVTMOW2t/meSJqvqFXnpjkm/GuJrP40m2VNXP9vdz\nqL+MrfmdjLE032sMpaq2Jnlfkje11n40temoxksfY0c7JofRWnugtXZma21j/z2/P5MJfX+ZlTCu\nlvsDgitxyWQmyZ9nMsPm/ct9Pifxff/DTC7l3p/kvr5cmslnFe5Ksrc/nt7bV5KP9X56IMnmqWP9\nepJ9fXnnVH1zkgf7Pv85A3yIdRH99vr8ZLbpKzP5hbcvyX9L8sJe/5n+fF/f/sqp/d/f++ORTM2S\nXE3jMMl5Sfb0sfXHmczEMq7m769/l+Rb/T3910xmABpbk3P/bCafBfybTP5BvepkjKX5XmMlL/P0\n1b5MPpd16Hf8x5c6XpYyJlfqMquv5mx/LD+ZsLDs48rXYwEADMRtUwCAgQhvAAADEd4AAAYivAEA\nDER4AwAYiPAGADAQ4Q0AYCD/H+NXjj8EaWOoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f8f5d7fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,5))\n",
    "sns.distplot(train.price.values,bins=50,kde=False)\n",
    "print(train.price.skew())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 created"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 这个属性由年月日组成"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train['created'] = pd.to_datetime(train['created'])\n",
    "test['created'] = pd.to_datetime(test['created'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count                   49348\n",
       "unique                  48671\n",
       "top       2016-05-14 01:11:03\n",
       "freq                        3\n",
       "first     2016-04-01 22:12:41\n",
       "last      2016-06-29 21:41:47\n",
       "Name: created, dtype: object"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.created.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count                   74659\n",
       "unique                  73091\n",
       "top       2016-05-14 05:10:46\n",
       "freq                        4\n",
       "first     2016-04-01 22:23:31\n",
       "last      2016-06-29 21:55:35\n",
       "Name: created, dtype: object"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.created.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上数据来看年份相关的时间属性， 基本上都挨得很近，横向对比没有参考价值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 特征工程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "先把数值型的特征选出来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "numerical_feature = train[[\"bathrooms\",\"bedrooms\",\"price\"]]  # 经度和纬度不知道怎么用，先不用了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 哑编码\n",
    "def get_dummies_cat(df):\n",
    "    \"\"\"对月份和周几进行哑编码\"\"\"\n",
    "    df_date = df[['month','dayofweek']]\n",
    "    df.drop(['month','dayofweek'], axis=1, inplace=True)\n",
    "    # print(\"Categorical features : \" + str(len(categorical_features)))\n",
    "    # df_cat = df[categorical_features]\n",
    "    # Create dummy features for categorical values via one-hot encoding\n",
    "    # print(\"NAs for categorical features in df : \" + str(df_cat.isnull().values.sum()))\n",
    "    df_date = pd.get_dummies(df_date,dummy_na=True)\n",
    "    # print(\"Remaining NAs for categorical features in df : \" + str(df_cat.isnull().values.sum()))\n",
    "    df = pd.concat((df, df_date), axis=1)\n",
    "    return df\n",
    "\n",
    "def numberical2cat(df):\n",
    "    \"\"\"先将月份和周几对应数值转换为category类型,再调用哑编码\"\"\"\n",
    "    df.replace({\"month\" : {4 : \"April\", 5 : \"May\", 6 : \"June\", 7 : \"July\",},\n",
    "                       \"dayofweek\" : {0 : \"Mon\", 1 : \"Tus\", 2 : \"Wed\", 3 : \"Thr\", 4 : \"Fri\", 5 : \"Sat\", 6 : \"Sun\"}\n",
    "                }, inplace = True)\n",
    "    df = get_dummies_cat(df)\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def create_newFeature(df):\n",
    "    df['num_feature']=df['features'].apply(len) # 特征数量\n",
    "    df['num_description_words'] = df['description'].apply(lambda x: len(x.split(\" \"))) # 描述词汇的数量\n",
    "    df = df.drop(['description'], axis=1) # 然后就不要description这个特征了\n",
    "    df['num_features']=df['features'].apply(len)  # 特征描述的单词数量\n",
    "    df['month'] = df['created'].dt.month  # 月\n",
    "    df['dayofweek'] = df['created'].dt.dayofweek # 周几，然后对月份和周几进行哑编码（因为它们的数值没有特别的意义）\n",
    "    # df['day'] = df['created'].dt.day  # 日\n",
    "    # df['hour'] = df['created'].dt.hour  # 时\n",
    "    df[\"room_num\"] = df[\"bedrooms\"] + df[\"bathrooms\"]  # room_num：bathroom房间数 + bedroom房间数\n",
    "    \n",
    "    managers_count = df['manager_id'].value_counts()\n",
    "    df['top_10_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 90)] else 0)\n",
    "    df['top_25_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 75)] else 0)\n",
    "    df['top_5_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 95)] else 0)\n",
    "    df['top_50_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 50)] else 0)\n",
    "    df['top_1_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 99)] else 0)\n",
    "    df['top_2_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 98)] else 0)\n",
    "    df['top_15_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 85)] else 0)\n",
    "    df['top_20_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 80)] else 0)\n",
    "    df['top_30_manager'] = df['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[managers_count.values >= np.percentile(managers_count.values, 70)] else 0)\n",
    "    \n",
    "    buildings_count = df['building_id'].value_counts()\n",
    "    df['top_10_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 90)] else 0)\n",
    "    df['top_25_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 75)] else 0)\n",
    "    df['top_5_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 95)] else 0)\n",
    "    df['top_50_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 50)] else 0)\n",
    "    df['top_1_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 99)] else 0)\n",
    "    df['top_2_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 98)] else 0)\n",
    "    df['top_15_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 85)] else 0)\n",
    "    df['top_20_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 80)] else 0)\n",
    "    df['top_30_building'] = df['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[buildings_count.values >= np.percentile(buildings_count.values, 70)] else 0)\n",
    "    \n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = create_newFeature(train)\n",
    "test = create_newFeature(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_withAddr = train[:]\n",
    "test_withAddr = test[:] # 先用两个变量将有地址的保存起来，万一之后模型效果不好，再来用这两个\n",
    "\n",
    "train.drop([\"display_address\", \"street_address\", \"manager_id\", \"building_id\"],axis=1, inplace=True)\n",
    "test.drop([\"display_address\", \"street_address\", \"manager_id\",\"building_id\"],axis=1, inplace=True)  # 现在先用不要地址的试试模型效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = numberical2cat(train)\n",
    "test = numberical2cat(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 提取y_train "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train = train['interest_level']   #形式为Class_x\n",
    "\n",
    "train = train.drop([\"interest_level\"], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train[\"features\"] = train[\"features\"].apply(lambda x:\" \".join([\"_\".join(i.split(\" \"))for i in x]))\n",
    "test['features'] = test[\"features\"].apply(lambda x: \" \".join([\"_\".join(i.split(\" \"))for i in x]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用CountVectorizer类来计算TF-IDF权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tfidf = CountVectorizer(stop_words =\"english\",max_features=200)\n",
    "train_sparse = tfidf.fit_transform(train[\"features\"])\n",
    "test_sparse = tfidf.transform(test[\"features\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(49348, 41)\n",
      "(74659, 41)\n"
     ]
    }
   ],
   "source": [
    "print(train.shape)\n",
    "print(test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from scipy import sparse\n",
    "train.drop(['created', 'features'], axis=1, inplace=True)\n",
    "test.drop(['created', 'features'], axis=1, inplace=True)\n",
    "train = sparse.hstack([train,train_sparse]).tocsr()\n",
    "test = sparse.hstack([test,test_sparse]).tocsr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(49348, 239)\n",
      "(74659, 239)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([  1.5   ,   3.    ,  40.7145, ...,   1.    ,   1.    ,   1.    ])"
      ]
     },
     "execution_count": 208,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(train.shape)\n",
    "print(test.shape)\n",
    "train.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. 建模 + 训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 由于数据量过大，运算时间过长，截取前10000行数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = train[:10000]\n",
    "y_train = y_train[:10000]\n",
    "#X_train = np.array(sub_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "#初始化特征的标准化器\n",
    "ss_X = StandardScaler()\n",
    "\n",
    "#分别对训练和测试数据的特征进行标准化处理\n",
    "X_train = ss_X.fit_transform(X_train)\n",
    "X_test = ss_X.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 default  Logistic Regression 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr= LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is:  [ 0.68005146  0.67267559  0.67953935  0.69576319  0.68873597]\n",
      "cv logloss is: 0.683353114283\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "loss = cross_val_score(lr, X_train, y_train, cv=5, scoring='neg_log_loss')\n",
    "print('logloss of each fold is: ',-loss)\n",
    "print('cv logloss is:', -loss.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 正则化的 Logistic Regression及参数调优（GridSearchCV）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） \n",
    "目标函数为：J = sum(logloss(f(xi), yi)) + C* penalty \n",
    "\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同：\n",
    "设置候选参数集合\n",
    "调用GridSearchCV\n",
    "调用fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#需要调优的参数\n",
    "# 请尝试将L1正则和L2正则分开，并配合合适的优化求解算法（slover）\n",
    "#tuned_parameters = {'penalty':['l1','l2'],\n",
    "#                   'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "#                   }\n",
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "lr_penalty= LogisticRegression()\n",
    "grid= GridSearchCV(lr_penalty, tuned_parameters,cv=5, scoring='neg_log_loss')\n",
    "grid.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([ 0.06457086,  0.1603363 ,  0.16214209,  0.19222279,  0.29659357,\n",
       "         0.21859179,  0.42041731,  0.23895068,  0.51176486,  0.26892538,\n",
       "         0.52284961,  0.30883694,  0.41309671,  0.22611156]),\n",
       " 'mean_score_time': array([ 0.00150647,  0.00149784,  0.00150833,  0.00150599,  0.00151186,\n",
       "         0.00132103,  0.0014153 ,  0.00179391,  0.00160012,  0.00139933,\n",
       "         0.00151892,  0.00170927,  0.00142422,  0.00150781]),\n",
       " 'mean_test_score': array([-0.75351813, -0.70413294, -0.70851804, -0.687973  , -0.68480471,\n",
       "        -0.68434687, -0.67805859, -0.68335225, -0.70180282, -0.68304724,\n",
       "        -0.7412286 , -0.68310304, -0.77364616, -0.68489118]),\n",
       " 'mean_train_score': array([-0.75281009, -0.70182211, -0.70671981, -0.68204137, -0.67486804,\n",
       "        -0.67409428, -0.65547061, -0.67308705, -0.64307725, -0.67128261,\n",
       "        -0.64087775, -0.67043055, -0.64064757, -0.67272011]),\n",
       " 'param_C': masked_array(data = [0.001 0.001 0.01 0.01 0.1 0.1 1 1 10 10 100 100 1000 1000],\n",
       "              mask = [False False False False False False False False False False False False\n",
       "  False False],\n",
       "        fill_value = ?),\n",
       " 'param_penalty': masked_array(data = ['l1' 'l2' 'l1' 'l2' 'l1' 'l2' 'l1' 'l2' 'l1' 'l2' 'l1' 'l2' 'l1' 'l2'],\n",
       "              mask = [False False False False False False False False False False False False\n",
       "  False False],\n",
       "        fill_value = ?),\n",
       " 'params': [{'C': 0.001, 'penalty': 'l1'},\n",
       "  {'C': 0.001, 'penalty': 'l2'},\n",
       "  {'C': 0.01, 'penalty': 'l1'},\n",
       "  {'C': 0.01, 'penalty': 'l2'},\n",
       "  {'C': 0.1, 'penalty': 'l1'},\n",
       "  {'C': 0.1, 'penalty': 'l2'},\n",
       "  {'C': 1, 'penalty': 'l1'},\n",
       "  {'C': 1, 'penalty': 'l2'},\n",
       "  {'C': 10, 'penalty': 'l1'},\n",
       "  {'C': 10, 'penalty': 'l2'},\n",
       "  {'C': 100, 'penalty': 'l1'},\n",
       "  {'C': 100, 'penalty': 'l2'},\n",
       "  {'C': 1000, 'penalty': 'l1'},\n",
       "  {'C': 1000, 'penalty': 'l2'}],\n",
       " 'rank_test_score': array([13, 10, 11,  8,  6,  5,  1,  4,  9,  2, 12,  3, 14,  7]),\n",
       " 'split0_test_score': array([-0.7490684 , -0.70004828, -0.70517638, -0.68396914, -0.68001101,\n",
       "        -0.68254049, -0.67864027, -0.68005146, -0.71659179, -0.68003042,\n",
       "        -0.78294245, -0.68022839, -0.82004203, -0.68391182]),\n",
       " 'split0_train_score': array([-0.75384792, -0.70301246, -0.70752653, -0.68500213, -0.6757537 ,\n",
       "        -0.67945654, -0.65565595, -0.67381598, -0.64258855, -0.67381305,\n",
       "        -0.64024404, -0.67153995, -0.63998309, -0.67864263]),\n",
       " 'split1_test_score': array([-0.75509401, -0.70236431, -0.70724141, -0.68171238, -0.67776064,\n",
       "        -0.67521129, -0.67028211, -0.67267559, -0.70031054, -0.67312307,\n",
       "        -0.75308478, -0.67206933, -0.80931258, -0.67300298]),\n",
       " 'split1_train_score': array([-0.7522171 , -0.70253481, -0.70744763, -0.68191664, -0.67619557,\n",
       "        -0.67548264, -0.65725795, -0.67168697, -0.64479742, -0.67210903,\n",
       "        -0.64247217, -0.66901014, -0.64211847, -0.6719325 ]),\n",
       " 'split2_test_score': array([-0.7470445 , -0.69743305, -0.70298512, -0.68125598, -0.68034991,\n",
       "        -0.67972807, -0.67347776, -0.67953935, -0.68627147, -0.67788842,\n",
       "        -0.71087141, -0.67920476, -0.73240212, -0.68180805]),\n",
       " 'split2_train_score': array([-0.75447251, -0.7035721 , -0.70828593, -0.68280789, -0.6764988 ,\n",
       "        -0.67475074, -0.65702459, -0.67793012, -0.6451343 , -0.67173158,\n",
       "        -0.64290571, -0.67283798, -0.64277707, -0.67610772]),\n",
       " 'split3_test_score': array([-0.76600489, -0.71505835, -0.71721696, -0.70014678, -0.69758065,\n",
       "        -0.69492893, -0.69042326, -0.69576319, -0.70960532, -0.69545786,\n",
       "        -0.73422903, -0.69525606, -0.75361816, -0.69713785]),\n",
       " 'split3_train_score': array([-0.75009807, -0.69887422, -0.70416235, -0.67957707, -0.67176207,\n",
       "        -0.6685857 , -0.6519955 , -0.67113533, -0.63886213, -0.66802153,\n",
       "        -0.63670339, -0.66801144, -0.63658105, -0.66654732]),\n",
       " 'split4_test_score': array([-0.7503795 , -0.70576357, -0.70997273, -0.69278512, -0.68832548,\n",
       "        -0.68932898, -0.67746895, -0.68873597, -0.69622479, -0.68874078,\n",
       "        -0.72498637, -0.68876091, -0.75282231, -0.68859755]),\n",
       " 'split4_train_score': array([-0.75341483, -0.70111696, -0.70617662, -0.68090314, -0.67413004,\n",
       "        -0.67219578, -0.65541907, -0.67086685, -0.64400383, -0.67073789,\n",
       "        -0.64206344, -0.67075326, -0.64177818, -0.67037037]),\n",
       " 'std_fit_time': array([ 0.01083068,  0.01244433,  0.02054682,  0.00863415,  0.07487779,\n",
       "         0.01883327,  0.21215434,  0.01597791,  0.17489742,  0.03194277,\n",
       "         0.26948454,  0.06259527,  0.10982945,  0.02048802]),\n",
       " 'std_score_time': array([  1.19462698e-05,   3.09771512e-04,   3.17058511e-04,\n",
       "          3.26563020e-04,   1.66021524e-05,   2.47257902e-04,\n",
       "          1.92368509e-04,   3.89224721e-04,   2.02523390e-04,\n",
       "          3.74838636e-04,   1.25720538e-05,   5.01182789e-04,\n",
       "          1.97218062e-04,   3.16821797e-04]),\n",
       " 'std_test_score': array([ 0.00678184,  0.00611116,  0.00492345,  0.00737155,  0.00732215,\n",
       "         0.0069978 ,  0.00685607,  0.00803033,  0.01052854,  0.0080079 ,\n",
       "         0.02496283,  0.008061  ,  0.03452928,  0.00794362]),\n",
       " 'std_train_score': array([ 0.00154357,  0.00168372,  0.00144699,  0.00182962,  0.00175487,\n",
       "         0.00360725,  0.0018827 ,  0.00263383,  0.00228277,  0.00190887,\n",
       "         0.00227541,  0.00173207,  0.00223404,  0.00426139])}"
      ]
     },
     "execution_count": 218,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.678058588937\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAEKCAYAAAArYJMgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd4VFX6wPHvO+mNNBJ6SUKTjnQE\npS5FpEgRVBAEseC6qz9dcXfV1S3iupZVVxEQBAGxIoiAFEGRIk1AVHrvkAYhPXN+f9xJSEICE5LJ\nJOH9PM88c8u5975DmXfOPeeeI8YYlFJKKVexuTsApZRSFZsmGqWUUi6liUYppZRLaaJRSinlUppo\nlFJKuZQmGqWUUi6liUYppZRLaaJRSinlUppolFJKuZSnuwMoCypXrmzq1q3r7jCUUqpc2bp163lj\nTMS1ymmiAerWrcuWLVvcHYZSSpUrInLEmXJ660wppZRLaaJRSinlUppolFJKuZQmGqWUUi6liUYp\npZRLaaJRSinlUppolFJKuZQmGqWUUi6liUYppYpi5u3WqyIopc+iiUYppZRL6RA0Sqkbmz0L0pMg\nLSnX+0VIv3R5Ofe+8/sAO3w2DjBg7GBMvmUK2Z5r2dgd6ybXvvzb7YUcz1WuXdA1KPi8yXEQVM3l\nf8SaaJRSrpd9e2bs18U/lzFWEsidFPIniUKTRhKkXcybWDKSnbywgE8QZKaC2ODkNmub2EAc70i+\nZQrZ7jgm/7LNBuJZeJkrzsVVrl3INUQuL+/9Brz9i/93cg2aaJRSrmcM2DMh9oD15V5YbeGK9fyJ\nwZFgMM5d1ysAfALBO9DxHgSVqudaD7SSh3cgeAdcXs6/zycQvPytL+iSTJruVkptTZpolFKukZYE\nh76H/SvgxBbISoO3br76MZ5+VyaGwEjwjrryi/9a614BVg1BFa6UkqUmGqVUyTAGzu2GfSus5HJk\nA9gzLtcWfKtD10mF1x68A8GjHHwlVYSaTCkrB3+rSqkyK/UCHPoO9q+EfSvhwnFre2Rj6PAw1OsJ\ntTvCh4Ot7S1GuC9W5TaaaJRSzjMGzv7qqLWshKMbrLYX7yCIvg1ue8pKLsE18x6ntYAbmiYapdTV\npSbCwTVWYtm/Ci6csLZXaQodH4X6vaBmO/D0dmuYquxyS6IRkTDgY6AucBgYboyJL6BcbWA6UAur\nm0k/Y8xhEfkAuA1IdBQdY4zZLiIC/BfoByQ7tm9z7adRqoIxBs7sulxrOfajVWvxqQTRXa12lno9\nrd5bSjnBXTWaScAqY8xkEZnkWH+6gHKzgX8aY1aISCBgz7XvKWPMZ/nK9wXqO17tgXcd70qpq0lJ\ngIOrrXaW/Ssh6bS1vWoz6PSYo9bSFjy83BunKpfclWgGAl0dy7OANeRLNCLSGPA0xqwAMMYkOXne\n2cYYA2wUkRARqWaMOVVSgStVIdjtcHqn43bYSji2CUwW+AZDTHerxlKvJwRVdXekqgJwV6Kpkv3l\nb4w5JSKRBZRpACSIyBdAFLASmGSMyXLs/6eIPAescmxPA2oAx3Kd47hjmyYapZLj8tZaLp21tldr\nAZ0ft2otNdqUjy7Gqlxx2b8oEVkJFPRz6C9OnsIT6AK0Ao5itemMAd4HngFOA97AVKza0ItYAz7k\nV+AjxCIyAZgAULt2bSdDUqocsdvh1PbLtZbjm60xrnxDoF4PqNfLqr0EVXF3pKqCc1miMcb0LGyf\niJzJvqUlItWAswUUOw78ZIw56DjmS6AD8H6uW2FpIjITeDLXMbVynaMmcLKQ+KZiJSnatGnj5HgW\nSpVxyXFw4FurIf/AKrh0ztpevRV0edJRa2kNNg/3xqluKO6qIy8C7gMmO94XFlBmMxAqIhHGmHNA\nd2ALQK4kJcAgYFeu8z4qIvOxOgEkavuMqtDsdjj5k/Uk/r4VcGIrYMAvLG+tJTDC3ZGqG5i7Es1k\n4BMRGYd1W2wYgIi0AR4yxow3xmSJyJPAKkdC2QpMcxw/V0QisG6VbQcecmxfgtW1eT9W9+axpfWB\nlCo1l85bz7PsX2nVWpJjAbFqKrc9bdVaqrfSWosqM8QYvWvUpk0bs2XLFneHodSVZt5uPdfS64XL\ntZaTPwEG/CvnrbUEhLs7WnWDEZGtxpg21yqn3UuUKouMsbocn98DKfHwfk9rTpEabaDrM1C/J1Rr\npaMTq3JBE41SZUlWJvy2CDb8zxpa3+YBfuHQ5yWr1uIf5u4IlSoyTTRKlQWpibDtQ/hxCiQeg7Bo\n6Pcf+PlzK9k0G+ruCJW6bppolHKn+CPw43uwbbY1u2SdztD339Cgj3VbrN0D7o5QqWLTRKOUOxzb\nDBvetm6TiQ2a3AkdH7F6iylVwWiiUaq0ZGXC7sVW+8vxTda4Yp0eg3YTILiGu6NTymU00SjlaqkX\n4Kc58OO7kHAUQqOg7yvQ8m5rGmOlKjhNNEq5SsLRy+0vaRegdifo/RI07KsPU6obiiYapUra8a1W\n+8uvjpGVmgy22l9qtHZvXEq5iSYapUqCPQt2f221vxzbCD7B0HEitH8Qgmu6Ozql3EoTjVLFkXYR\nfpoLG9+BhCMQWtfqntzybvAJcnd0SpUJmmiUuh4Jx2DTe7B1NqQlQu2O0Puf0LCftr8olY8mGqWK\n4sRW6/bYL19a600GQYeJUFPbX5QqjCYapa7FngV7llgJ5ugG8KlkNe63exBCal37eKVucJpolCpM\nWhJsd7S/xB+GkNrQZzK0ulfbX5QqAk00SuWXeMLR/vKBNdhlrfbQ60VoeDt46H8ZpYpK/9cole3k\nT472lwVg7NB4oNX+UqutuyNTyiWOjBoNQJ0PZ7v0Oppo1I3NngV7l1kJ5sg68A6C9g9Z44+F1nF3\ndBVGaX2hXS9jDGRkYDIzL78yMiHzym2nnn0WgBqvvootIMB6+fshOgldoTTRqBtT+iXYPs9qf4k7\nCMG1ofe/oNUo8K3k7ujKNWMMJjUVe2oqJjkZe0oKWZcugd3OpQ0bcr6wTWYG5PoCt77Mr7ItI9cX\nvtPbHMki/7Z8SYWsrCJ/zoP9+l1eEcHm748tMNBKPIGB2AL88QgMxOYfcHl7QAC2QOvdI0/ZAGwB\ngRU2aWmiURXOVX89XzgJm6bClpmQmmBNjTzsOWh0R5lsf1nacxAAfVd+WWLnNHa7lQhSUrCnpGAc\n7/aUVOwpyY51x3JqKvZkR7nUFGs51bEv93JK3vMV5ujY+4sesM2GeHoinp7g5ZWzfHmbJ+KZb7uv\nDzbPwMvrXp7gma9c/m1ejvNdbZuXJ+defwOA8AcfxJ6UhP3SJeyXrPespCTsl5Jztqefj7W2X7qE\n/dIlyMy89ufNTlq5k1B2cgoILHB7oYmrjCQtt/zPEpEw4GOgLnAYGG6MiS+gXG1gOlALMEA/Y8xh\nEVkLZHf7iQQ2GWMGiUhXYCFwyLHvC2PMiy78KKq8OLndqr3s+txqf7npDuj4KNRq5+7IAMetm5xf\n8hk5L8/MdMQYUnbsuEpiyP6Sv/IL355yZWIwqalFjk98fLD5+SH+fth8/bD5WS+PkBC8qlXD5ueL\n+Plh8/O/vOzrh83fKnfu3SmICFWf/WueL+2cZJFvPU9SKQNflLnFzfwAgOD+txfpOGMMJj39cnJK\nKjg5FZa0Ms7HklYSScuRoDwCAkg/cgRbcPB1/CkUjbt+wk0CVhljJovIJMf60wWUmw380xizQkQC\nATuAMaZLdgER+RwruWRba4zp77rQVXlhsrLgtyWY9e9iDq3HeAZiGt+LaXIXxj8Sk5SB2bnT+nJP\nz8jzBX/5lZ5nndyJoNBjcr3yJI5858p3fEGyR0k7fNeIq35WcXzx23x98yQDj8rheDmWxc/XkQj8\nrkgGeZZ9fa0vp+zz+RX/V3H8R/MB8G9b/jtWXG87k4hYCdvHB8LDixVDsZNWXBxpSUlkxsbi6en6\nNOCuRDMQ6OpYngWsIV+iEZHGgKcxZgWAMSYp/0lEJAjoDox1YayqDDMZGaQdPETant2k7tlD2u49\npGzbjLEbdjdpmqtkNcf7csereMTbG/Hyynnh5ZlrPe8+m79/nvWcl6fjGO8r9+F4//m1dzBio/VL\nz11OJn7ZicPx7uNT5n7151dWOwGUVyWVtLJvM7uauxJNFWPMKQBjzCkRiSygTAMgQUS+AKKAlcAk\nY0zuVrvBWDWjC7m2dRSRHcBJ4EljzC+u+QiqtGXGxZG2ezepe/Y63veQduAAOGoD4uWFd+2qBNZI\nwTsgEwmuikR1RKo3t76MC/kyz/vKThKeBScHLy/w8EBESuUzX3pvLgCBt91WKtdTyhVclmhEZCVQ\ntYBdf3HyFJ5AF6AVcBSrTWcM8H6uMiOx2nCybQPqGGOSRKQf8CVQv5D4JgATAGrXru1kSKo0mIwM\n0g4dIm3PXqumsnsPaXv2kHnuXE4Zz4gIfBo1IrBLZ3waNMS3QT28j36KrHuVjBRPYvdXpurMzVBK\nCcFVUuWYu0MoEWOXWTcdZvaZ6eZIiq8ifZbS4rJEY4zpWdg+ETkjItUctZlqwNkCih0HfjLGHHQc\n8yXQAUeiEZFwoB1WrSb7mhdyLS8RkXdEpLIx5nwB8U0FpgK0adPGXM9nVMWXGR9P2p49pO7eTdqe\nvaTu2U36vv05bRbi5YV3vXoE3HILPg0b4tuoIT4NG+IZFnb5JAnH4IsHrHHIWt7DqXfXYey2cp9k\nACbfXRfI9Y9cqXLIXbfOFgH3AZMd7wsLKLMZCBWRCGPMOay2mC259g8DFhtjcrrQiEhV4IwxxohI\nO8AGxLroM6giMJmZpB8+nFM7Sd2zm7Tde8g8e/k3hkdEZXwbNCRg9Ch8GzXCp0FDfKKjrNtVhfnt\nK1j4qPXg5Z3ToPlwzOelc9/ZlVLSszh4Pom0pFp4+Z9ydziqgvrbPdaUFq6um7kr0UwGPhGRcVi3\nxYYBiEgb4CFjzHhjTJaIPAmsEuuG+FZgWq5zjHCcJ7ehwMMikgmkACOMMVpbKWVZCQlWQtm7x3rf\nvZu0/fsx6elWAS8vfKKjCejYAZ+GjfBp2ADfRo3wLEqjZkYKLP8rbJ4O1VvB0BkQFg3A8O6XAPix\npD9YCTPGcPZiGgfOJnHgXBIHzl3iwLkkDp67xImE7GdRemHzSOHDjUcY0bYWXh5lu9FfqYK4JdEY\nY2KBHgVs3wKMz7W+AmheyDm6FrDtbeDtEgtUXZXJyiL98GHHrS9HLWXPXjJPn84p4xEejm/DhoTe\nc49126tRI3yiohBv7+u/8Lk98OlYOPuL9SxMj+fBsxjnc7HUjCyOxCZbyeRsEgfPX04oSWmXn4Xw\n9/YgOiKANnVDGV65FjGRAfz1+8lcim3Os1/uYuYPh/hTn4b0blK11DojKFUSyt6j0KpMykpMzOk+\nnJ1Q0vbtw6SlWQU8PfGJisK/bVtHO0ojfBs2wDMiouSCMAa2zYalT4N3ANzzGdTvVXLnLwZjDOeT\n0jmYp2ZiLR+LTyZ3vbp6sC/REYEMubkGMZGBRFcOJCYygKqVfK9IIH/fegJv/xO81PEdJi/dzUNz\ntnFz7RCe6XcTbeuGoVR5oIlGAfDN7dYT8r9btIH0I0fzPJeSumcPmacutxN4hIbi06ghoSNH5jTQ\ne8fEYCtOLeVaUhPhqz/CL19A1G1w51QIKqhTo2ulZ9o5GneJ/WcvcfB8EgfOXk4qF1Iv1058PG1E\nRwTSvGYwg1rVICYigJiIQKIqBxDgU7T/diLQ46Yq3NYggs+3Hee1FXsZNmUDvRpX4ek+jagXGVjS\nH1OpEqWJRpEZH0+luDT8kjPZ07rN5SFKPDzwiY7C/+ab8WnU0Gqgb9gQz4iI0r11c3wLfHY/JB6H\nHs/BLY+Dix9QjL+U7mg3sW5xZbehHI1LJst+uXoSGeRDTEQgA1pWd9RMAomJCKB6sB82W8n+GXl6\n2LirbW0GtKjBjHWHeHfNAXq/8T3D29Ti8Z71iazkW6LXU6qkaKK5gaUfPUrcB7NI+OILQlLTSfX1\nIPSu4TkN9D716llPHruL3Q7r/wvf/gOCqsP9y0p0bLLMLDtH45JzJZLLSSU++fKQMN4eNupW9qdR\n1SBub1aNaEftJDoigCDfq/SIc4IxhkyTSUZWBhl265Vpz8xZtpMGGH6N/RVjDHZjx2C4pYmdJlEh\nfLb1GJ//+h0Ld39Pv2ZV6Ne8Kr6eNuzYrWFKsI6xm3zr2fuNyVnOLgfkKZN9zexzZK9nH3ut7QbD\n8YvHAXht62s5n12Qgpel4O25FVam0O25z5NnsYjnEeHUpVMIwvzd8/Hz9MPX0xc/T7+cl6+HL35e\njnfHfpvc2J04NNHcgFK2byd2xkwurliBeHpSacAd7N60jAxvD1o984y7w7NcPAMLHoSDq6HxILjj\nv+AX4tShBgMYLqZfJMOeQdylFA6cT+BQ7EUOxyZyNO4ixxOTOH0hiSx7JkgWIlkE+QmRlbxo0tCD\n8EAPQgM8CPYXAnyELJOZkwT2ZWbw64kMMo5lkJGVNzHkvLKuTBq5t+fed1WO76e7Ft9VaBGfWtb7\nN3HwzRqn/ohKhU1sCIKIkGW3BvSY99s8wDGIqIP193Xlct7FQsoXcp7S8M8f/+l0WV8P35yElPPu\nSEh+Hn55txeUuArYnnubp61sf5WX7ehUiTF2O0mrVxM7YyYpW7diq1SJ8AceIPTee/CKjOQvy04C\n0NvNcQKwfyUseAjSkuCO/2JvNYrE9AvEJRwgLjWO+NT4PO9xqXHEp8UTnxpPbEosKRIPAp0+6lT4\nNYLBJ9+gtZlY4xadTAVSgXyP+XqIB142L+vl4YWnzTNnPWfZw1r39fQlyBZ0xfbCyhe2/a9rXgeE\n//SYZH1xiyBIznL2l7lNbBw8e4k5G4/yy6kkqgf7cW+HOtwSE4HNdrlM9nvuJGATGzYc5xbJWc4u\nB+Qpk/s8uY9HwIYtZ3tu7nyavtCkdp3J7oHlD2CM4bVur5GSmUJqZmqe95SsFFIyUkjNyrfd8cop\nn5VKYmoiZ7LOXLG/qEnT0+ZpJR8PvytqU9nv/p7+eZOcpx/nks/h7+VfpGtdD000FZw9NZXELxcS\n98EHpB8+jFeNGlT5858JGXIntoAAt8SUZc8iMT0xb6JIjSc++TyxB5YTf3YX8eGViAuuT/z+mST8\n8nrOLZ38gn2CCfUJJdQ3FD+pSmpiFdKSfMDYqBkcSkSgP5FBAVSrFEj14ACqVgrE19M775d6vi/8\ngrZ72jzdcvvjkz2fANCtdrdrlm1bFYY3M6zec5bJS3cz+cskWtW+wDN9b6Jd1I3bQ63QW3HX2YRm\nEyupVvarXMzICmaMIS0rLW/iypewrpa8UjIuH5OSmUJcatwVyTDTXO64UtXf9Z1qNNFUUJnx8cTP\nm0f83HlkxcXh27QpNV5/jaBevay5PkpQduKIS7FqFrmTR/73+LR4EtISCk8cWVmEVoogLLwBUX6V\nudnXSiJhvmGE+YblWQ72CcbL5sW2o/G8tOQ3Nh+OJ7pyAGnei/AJPMK3979eop+zPBARujeqwm0N\nIvl8q9VDbfh7G+h5UxUm9W1Ivciga59EuZWI4Otp3WoLwbnbxUWVYc8gJTOFh1c8XCo/oDTRVDDp\nhw8TO2sWiQu+xKSmEti1K2H3j8W/bVune4pl2bNISEvISQyxqbHWcv4aiGM9IS2h0Kp+do0jzDeM\nqOAobva9+YqEEXpsC2HfTiYYwWvAW9B4oFNxHjyXxCvf7GTprtNUDvThH4OaclfbWnSe/ZbTf14V\nlYdNGN62Fne0qM6MdYeYsuYAv3v9e+5qW4s/9mxAFe2hdkPzsnnh5e2Ft0fpPOisiaaCSP7pJ+Jm\nzODiylVWA//AAYSPHYtPTMxVjzPGsPXMVg4nHiYpI4lb599atMRRpYDE4VgO8Qm5eiNl+iVY+if4\naQ7Uag9DpkPItUfSPncxjTdX7WPepqP4eNr4Y8/6PNAlusjPp9wI/Lw9mNitHiPb1eatb/cxZ+MR\nFvx0gge6RDPh1uhi95pTyhn6P7McM1lZXPz2W+JmzCTlp5+wBQcT/uAEwu6555pP5B+7eIyvDnzF\nogOLOJF0ApvYqORdiV51ehHquF0V7hues+xU4iiK0z9bz8ac3wddnoSuz4DH1c99KS2TaWsPMu37\ng6Rm2hnZrhZ/6NGAiCA3dsEuJ8ICvHn+jiaM7RTFK8v38Na3+5n341Ee61Gfke1q4+15Y3e/Va6l\niaYcsqekkPjll8R+8AEZR47iVbMmVf7yF6uB37/wHiRJ6UksP7KchfsXsu3sNgShfbX2TGw5kU/3\nfoqHePBsx2ddG7wxsGmaNSCmXyiMXgjRV5/UKyPLzsebj/HGyn2cT0qjb9OqPNW7IdER+kR8UdUO\n9+etka14oEsU/1ryG88v+oUZ6w7xp96N6NdMx1BTrqGJphzJjIsjfu484ufNIys+Ht9mzYh843WC\nevYstIE/y57Fj6d+ZOGBhXx79FtSs1KpW6kuf7j5D/SP7k/VAKvHyRf7vnD9B0iOs4b03/M11O8N\ng96BgMJ77hhj+OaXM/x72W4Onr9E27qhvDeqNa3rhLo+1gquec0QPnqgA2v2nGPy0t1MnLeNFrVC\neKZvIzpEF28+e6Xy00RTDqQdOkRcdgN/WhqB3boRPu5+/Fq3LvQX6MGEgyw8sJDFBxdzNvkslbwr\nMbDeQAbEDKBZ5Wal/8v18DprcrKks9D7Jejw8FUnJttyOI6Xlu5m65F4YiICmDa6DT1vitRf3CVI\nROjWKJJbs8dQW76XEVM30qNRJE/3bUSDKtpDTZUMTTRlWPK2bcTOmEHSqm8RLy+CBw4kbOwYfKKj\nCyyfkJrA0sNLWbR/Ebtid+EhHnSu0Zmn2z5N11pdS62HSR72LPj+FfjuZQiNgvEroXrLQovvP5vE\nv5ftZvmvZ4gM8uGlO5sxrHVNPIswD0vd9CdLIvIyoTQecPSwCcPb1GKAo4fau6sP0OeN7xnWuhaP\n92pA1eDi91BLPjKhBCItG3QK56LTRFPGmKwsLq5aZTXwb9+OR3Aw4Q89aDXwV77yNlOGPYO1x9ey\n6MAivjv+HZn2TBqGNuSpNk/RL7qfyx4qc0riCasWc2QdNB8Bt/8HfAr+lXz2Qiqvr9zHJ1uO4efl\nwZO/a8D9naPw9y76P9GPH+xY3MhvSL5eHjzStR4j2tbm7W/38+HGwyzccYJxnaN48LYYKmkPNXWd\nNNGUEfaUFBIWLCDug1lkHD2KV61aVHn2r4QMHnxFA78xht/ifmPRgUUsObiE+LR4wnzDGNloJANj\nBtIwrKGbPkUuu5fAwkcgMx0GvwctRhRYLCktk6nfHWDa2kNkZNkZ1aEOv+9ej/BA7UnmLmEB3jx3\nR2PGdKrLf5bv4X+rD/DRpmP8vns97mlfR3uoVSClVTvTRONmmbGxxM+dS/y8j8hKSMC3eXMin3iC\noF49EQ+PPGXPJZ/j64Nfs/DAQvYn7MfL5kXXWl0ZGDOQTjU64WUrA784M1JhxXOw6T2o1gKGzoTw\nK5/lyciy89Gmo/x35T5iL6Vze/NqPPW7htSt7J5hcdSVaof78+bIVjzQJZp/LfmNF776lZnrDvNU\n74b0b15N28uU0zTRuEnawUPEffABiV9+icnIILB7d8LvH4vfzTfn+Q+clpXG6qOrWXhgIetPrsdu\n7DSv3Jy/tv8rfaL6EJx/ZEh3Or/PmmL5zM/Q4RHo+TfwzFszMcaw5OfTvPLNbg7HJtM+Koz3+91E\ny1quGWpDFV+zmsHMe6A9a/ae4+Wlu/n9Rz8xfe1BJvW9iY4x2kNNXZsmmlJkjCFl61ZiZ8wk6dtv\nEW9vggcNImzMGHyio/KU23FuBwsPLOSbQ99wMeMiVfyrcH/T+xkQM4Co4KirXOX6FKsKbQxsnwtL\nngIvP7j7E2hw5TjQPx6M5aWlu9l+LIEGVQKZMaYN3RpqT7LyQETo1jCSW+tH8IVjls+R0zbSvVEk\nT/dpRMOq2kNNFc5tiUZEwoCPgbrAYWC4MSY+X5luQO6RERsBI4wxX4pIFDAfCAO2AaOMMeki4gPM\nBloDscBdxpjDrv00V2eysri4YiWxM2eQumMnHiEhVH7kEULvuRvP8Mu/CE8mnWTRgUV8deArjl48\nip+nHz1r92RAvQG0rdIWD5vHVa7iJqkXYPHjsOszqNsF7pwGlarlKbL3zEVeXrqbVbvPUrWSL/8e\n0pwhrWviUcIzUCrX87AJw9pYY6jNXHeYd9bsp+9/v2do65o83qsB1YL93B2iKoMk95wMpXphkX8D\nccaYySIyCQg1xjx9lfJhwH6gpjEmWUQ+Ab4wxswXkSnADmPMuyLyCNDcGPOQiIwABhtjCp81CmjT\npo3ZsmVLyX04B3ty8uUG/mPH8Kpdm7Ax91kN/H7Wf8jkjGSWH1nOogOL2Hx6MwBtq7ZlQMwAetXp\nRYBXGW6zOLHVGkYm4Rh0ewY6PwG5kuHpxFReX7GXT7ceI8Dbk4e7xTC2UxR+3mUwYarrEn8pnbdX\n7+fDDUcQgXGdo3io65U91O56bwOgPQIrGhHZaoxpc81ybkw0e4CuxphTIlINWGOMKbS7lIhMAG4z\nxtwj1r2Wc0BVY0ymiHQE/maM6S0i3ziWN4iIJ3AaiDBX+aDXm2gKm8wp8/x54ubOJWHeR2QlJuLX\nogVh4+4nqEcPxMMDu7Gz6fQmFu1fxMqjK0nJTKFWUC0GxAzgjpg7qBFYo8ixlCq7HTa8DategKBq\nMOR9qN0+Z/eF1Aze++4A7/9wiCy7YVSHujzavR5hAW54jkeVimNxyfxn+R4Wbj9JqL8Xv+9en3s6\n1MbH0/pRoYmmYnI20bizjaaKMeYUgCPZRF6j/Agge8LxcCDBmJzZe44D2d/ONYBjjvNmikiio3y+\n+RJLXtrBg8TN/IDEhQutBv4e3Qm//378b74ZgMOJh61bYwe/4vSl0wR6BdIvqh8D6w2kZUTL8tFW\nkXTWmv3ywCq4aQAMeNMaswxIz7QzZ+MR3vp2H/HJGQxoUZ0nf9eQ2uGun8FPuVetMH/+O6IV4ztH\nM3nZb7y4+Fdmrj/EU70b0b9ZtWufQFVoLk00IrISKGj6tr8U8TzVgGbAN9mbCihmnNiX+5wTgAkA\ntWtfe2j6gox46xcwhuTwzVZH3PUaAAAgAElEQVQD/+rViI8PwYMHEzbmPnyiokhMS+STPZ+w8MBC\ndp7biU1sdKzekSdaP0G3Wt3w9SxH84Ic+Ba+eBDSLsDtr0Gb+0EEu92w+OdT/OebPRyNS6ZTTDjP\n9L2JZjXLUI84VSqa1Qxmzrj2fL/vPC8t+Y3HHD3U0jPtBPuVge73yi2cSjQicguw3RhzSUTuBW4G\n/muMOXK144wxPa9yzjMiUi3XrbOzVznVcGCBMSbDsX4eCBERT0etpibWdO9g1W5qAccdt86CgbgC\nYpsKTAXr1tnVPkdhvFOzCD2fypFRo60G/okTCb17JIQGs/7kehaueZs1x9aQbk+nXkg9nmj9BLdH\n306k/7Uqb2VMVgas/if88AZENLRGXK7SGID1B84zeeludh5PpFHVID4Y25bbGkSUj9qZcgkR4bYG\nEXSuV5kFP53gteV7OJmYSoifFwnJ6YT46y3UG42zNZp3gRYi0gL4E/A+Vs+uq4/vfnWLgPuAyY73\nhVcpOxJ4JnvFGGNEZDUwFKvnWe7js8+7wbH/26u1zxSXzW6o+rfnCR44kH0pR/ngwAy+Pvg1samx\nhPqEMqzhMAbEDOCmsJvK55dv/GH4fDwc3wytx1gDYnr7s/v0BSYv3c2aPeeoHuzLq8NaMKhVDe1J\npnJ42IShrWvSv3k1er32HcfjUxg2ZQOzx7XT3mk3GGcTTabjy30gVk3mfRG5r5jXngx8IiLjgKPA\nMAARaQM8ZIwZ71ivi1VD+S7f8U8D80XkH8BPWMkPx/uHIrIfqyZT8NgnJSDd14O99fzZ0TyDRStH\nsztuN542T26reRsDYgbQpUYXvDzK8e2CXV/AV38ABIZ9AE0GczIhhdcW7uDzbccJ8vHkmb6NuK9T\nXXy9tCeZKpivlwfVQ/wI8PHkeHwKQ95Zz+xx7agXqc/e3Cic6nUmIt8By4CxwK1YPb62G2OauTa8\n0nG9vc4GfjmQg4kHAWgS3oQBMQPoG9WXUN/yN19Knl5B6cmw7GnYNhtqtoMh00n0rc67aw4wc90h\njIH7OtVhYrd6ehtEOSX739ez/RszZuZmMu123r+vrc4tVM6VdK+zu4C7gXHGmNMiUht4pTgBVgQB\nXgFU9a/Kuz3fpV5oPXeHUzLO/GINI3N+L3R+grQuT/PhppO8vXo1iSkZDG5Zgyd+14CaodqTTBVd\n0xrBfPFwJ0bP+JF7pm/knXtupnujKu4OS7mYs4nmItYtsywRaYD1hP5HrgurfPDx8KFmUM0KkWSe\nO/8kofY4mBoLfiHY7/2SRRfr88rr6zmRkEKX+pWZ1LcRTaprTzJVPLXD/fns4U6MnbmZB2ZvZfKd\nzRjWppa7w1Iu5Ox4398DPiJSA1iFdQvtA1cFpUqZ3U71rBNUzzoJUbfyY+9F3LHEgz9+vJ0Qfy8+\nHNeOD8e11ySjSkzlQB8+mtCBTjHhPPXZTt5dcwB3PTyuXM/ZRCPGmGTgTuAtY8xgoInrwlKlxhj4\n5hlC7fHslbqMTv0/7pp7gITkDN64qyVfPdqZLvUj3B2lqoACfTx5/762DGhRnZeX7ebFxb9it2uy\nqYicvXUmjmFe7gHGObZpN6OKYPU/4ccpfORxB89cGkHwiYv89fabGNWxTs7wIUq5irenjTfuakl4\noDcz1x0mNimd/wxroZOrVTDOJpo/Yj3HssAY84uIRAOrXReWKhU/vAHfv8LPkQN55uhwKgf6sOr/\nuuoT3KpU2WzCc/0bExnky8vLdhN3KZ0po1oT6KOzmFQUTv1sMMZ8Z4wZALwjIoHGmIPGmMdcHJty\npc3vw8rn2RvxOwYeHUZEoC/RlQM0ySi3EBEe7hrDK0Obs+FgLCOnbuR8Upq7w1IlxNkhaJphjQQQ\nZq3KOWC0MeYXVwZX1pXWfNslbsfH8PX/caTyrfQ7di8DW9XiRHxy+Ry5QFUow9rUIjzQm0fmbmPo\nu+uZfX97HZS1AnD2Ruh7wBPGmDrGmNrA/wHTXBeWcpnfFsOXD3MqrC2/O34/vZvV4pWhzTXJqDKj\ne6MqzB3fgYSUDO58dz2/nEx0d0iqmJxNNAHGmJw2GWPMGqAMz8ilCnTgW/hsLOeDm9Dj5EPc2rgW\nb4xoiaeHNrwq1/r4wY5FmoumdZ1QPnuoI94ewl3vbWT9AZfP8qFcyNlvmIMi8qyI1HW8/goccmVg\nqoQd3Qjz7yExoC49zjxK2wa1ePvuVnhpklFlVL3IID5/pBPVQ3wZM2MzX+885e6Q1HVydqyzUOAF\noDPWfC/fY81iGe/a8EqHq6ZyLjNObodZd5DkFUb32KeJiYpm5ti2OhCmKhcSkzMYN2szW4/G88KA\nJozuWNfdISmHEh3rzJFQtJdZeXRuD8y5kxSPQPrGPUmdOnV5f0wbTTKq3Aj292LO+PY8Ou8nnlv4\nC+cupvFErwbarliOXDXRiMhXFDA7ZTZHl2dVVsUfhtkDSbPb6H/xKcJqRDNjTFv8vfX5BFW++Hp5\nMOXem/nLgl289e1+zl1M4x+Dmmr7YjlxrW+c/5RKFKrkXTgJswaQkZbCkOS/4FulPrPHtiPIV5+T\nUeWTp4eNyUOaERHkw9ur93M+KZ23725V6rXzPFNqKKdcNdEYY/JPNqbKg0uxMHsQWUnnuTvtz2SE\nN+Kjce0J9tcko8o3EeHJ3g2JCPLhb1/9wqj3f2T66Lb6b7uMc6reKSI/i8jOfK+1IvK6iIS7OkhV\nBKmJMGcw9vjDjMt4itiQpswZ356wAJ2gTFUc93Wqy1sjW7HjWCLD3lvP6cRUd4ekrsLZG5xLga+x\nBtW8B/gKWAucRqcLKDvSL8Hc4djP/MrErCc4GNCSeeM7EBHk4+7IlCpx/ZtX54OxbTmZkMqQd9ez\n/2ySu0NShXA20dxijHnGGPOz4/UX4DZjzMtAXdeFp5yWmQYf34s5vomnze/Z6duOeQ+0p2qwr7sj\nU8plOtWrzPwJHUjLtDN0ynq2Ha0QT1xUOM4mmkARaZ+9IiLtgEDHamaJR6WKJisTPrsfDnzLi/IQ\n33vdwrwH2ut0y+qG0LRGMJ8/3JFgPy/unraR1bvPujsklY+ziWY8MF1EDonIYWA6MF5EAoCXinpR\nEQkTkRUiss/xHlpAmW4isj3XK1VEBjn2zRWRPSKyS0RmiIiXY3tXEUnMdcxzRY2t3LHbYeFE2L2Y\nVz3u5ytbD+Y90IE64TpCkLpx1AkP4LOHOlEvMpDxs7fw2dbj7g5J5eLsNAGbjTHNgJZAS2NMc8e2\nS8aYT67jupOAVcaY+lhTQ08q4JqrjTEtjTEtge5AMrDcsXsu0AhoBvhhJcJsa7OPM8a8eB2xlR/G\nwNKnYOd83vO8mzmmL3PHtycmIvDaxypVwUQE+TB/Qkc6RIfx5Kc7mPKdTg9dVjjb6yxYRF7DSgor\nReRVESnOBPIDgVmO5VnAoGuUHwosdUwnjTFmiXEANgE1ixFL+bXqBdg8nXmeg/lf5kA+HNeehlWD\n3B2VUm4T6OPJjDFt6d+8GpOX7ubvi3/T6aHLAGdvnc0ALgLDHa8LQHEmY6lijDkF4HiPvEb5EcBH\n+Tc6bpmNApbl2txRRHaIyFIRaVKMGMu2ta/CD6+zyLMP/8oYwexxHWhaozi5X6mKwcfTgzdHtGJM\np7rMWHeIP368nfRMu7vDuqE5OxZJjDFmSK71F0Rk+9UOEJGVQNUCdv3F2eAc56mGdYvsmwJ2vwN8\nb4xZ61jfBtQxxiSJSD/gS6B+IeedAEwAqF27dlFCcr8fp8KqF1np1ZVJqfcxa1w7WtYKcXdUSpUZ\nNpvw/B2Niazkw7+X7SE+OZ1379Xpod3F2RpNioh0zl4RkVuAlKsdYIzpaYxpWsBrIXDGkUCyE8nV\nuokMBxYYYzJybxSR54EI4Ilc17xgjElyLC8BvESkciHxTTXGtDHGtImIiLjaRylbts+DpU+x3qsD\nj6U8wPT72tG2bpi7o1KqzBERHulaj38Pbc76A7HcPU2nh3YXZxPNw8D/ROSwiBwB3gYeKsZ1FwH3\nOZbvAxZepexI8t02E5HxQG9gpDHGnmt7VXEM6erogm0DYosRZ9ny60LMwon85NWKB5In8s6odnSq\nV2AeVUo5DG9Ti6mjWrP3zEWGvrueY3HJ7g7phuNsr7PtxpgWQHOgmTGmlTFmRzGuOxnoJSL7gF6O\ndUSkjYhMzy4kInWBWkD+MdemAFWADfm6MQ8FdonIDuBNYISpKN1O9q3EfDaOPZ6NGH3pMd64pz1d\nG16raUspBdDjpirMHd+e+GSdHtodrjrxmYg8UehOwBjzWolH5AZlfuKzw+swc4ZwWGowKOlpXrq7\nC/2aVXN3VEqVO/vOXGT0jE0kpWYydXQbOsYUfahGHb35MmcnPrtWjSboGi/laie2YeYN5xSVGZr0\nJC8Mv0WTjFLXqX6VID5/uBNVg325b8Ymlvys00OXhmtNE/BCaQWiCnDmV8ycOzlvD+TOpD/x9JDO\nDGpVw91RKVWuVQ/x49OHOjJu1hYmztvGiwOaMEqnh3apIk9PJyLbXBGIyif2AObDQSSm2xhy6Wkm\nDryV4W1ruTsqpSqEEH9v5oxrT/eGkTy78BdeW75HRxFwoeuZB1Un6na1xBOY2QO5lJLCsOSnGd2v\nq/7iUqqE+Xl78N6o1gxvU5M3v93PnxfsIjNLH+x0het5eunrEo9CXZZ0DjN7IGkXYxmR8mcG/a4H\n47tEuzsqpSokTw8bLw9pTkSQD/9bfYDYpDTeHFn600NXdEWu0Rhj/uqKQBSQEo/5cBAZcUcZlfIk\n3bv3ZmK3eu6OSqkKTUR4qncj/nZHY1b8dobR728iMSXj2gcqpzk7qOZFEbmQ73VMRBaIiP7cLglp\nSZi5w7Gf3c34tD9y8639eLxngaPnKKVcYMwtUbw5ohU/HYtn+JQNOj10CXK2RvMa8BRQA2uk5CeB\nacB8rAE3VXFkpML8uzHHt/BI2qNEdxjIpD6NcAxyoJQqJXe0qM7MMe04Hp+s00OXIGcTTR9jzHvG\nmIuO8cSmAv2MMR8DV0xapoogKwM+GwuHvuPJ9AmEtRnK83c01iSjlJt0rl+Zjx/sSFpmFsOmrOcn\nnR662JxNNHYRGS4iNsdreK592ifwetmz4MuHYc8Sns0Yg7S8m38OaqpJRik3s6aH7kSQrxd3T/uR\n1Xt0eujicDbR3IM178tZ4Ixj+V4R8QMedVFsFZsx8PUT8POnvJwxgoSmY/j30ObYbJpklCoL6oQH\n8PnDnYiOCGD8rC18rtNDXzenujcbYw4CdxSy+4eSC+cGYQyseBa2fsD/MgdwsNEDvD28BR6aZJQq\nU6zpoTvw0Jyt/N+nO4i9pNMMXA9ne501EJFVIrLLsd5cRLSb8/X6/j+w/i1mZ/Via8zveWvkzXh5\nXM+zs0opVwvy9WLGmLbc3rwa/1qymyOxyRVmFIG73tuQM0ioKzn77TYNeAbIADDG7MSaXlkV1cZ3\nYfU/+DyrCytq/x/v3Nsab09NMkqVZT6eHrzlmB769IVU9p+7RFJaprvDKjec/YbzN8ZsyrdN/5SL\natuHsGwSy+zt+LTGJKbe106fQFaqnMieHrpWqB9xl9IZ8NYPOq+Nk5xNNOdFJAZHDzMRGQro+NpF\nsesLzFeP8b29Oe9X+QvTx3bAz1uTjFLliYhQPcSPm6oGcSk9k8HvrOfDjUcqzK00V3E20UwE3gMa\nicgJ4I8UbyrnG8ve5dg/f4At9ga8Wfl5pt9/C4E+1zPMnFKqLKjk58WSx7rQMTqcZ7/cxaPzfuJC\nqg5bUxhnE80JYCbwT6zRAFYA97kqqArl0FrsH9/Lr/bavBTyN6aNu5VgPy93R6WUKqbwQB9mjmnL\npL6NWPbLafq/+QM7jye4O6wyydlEsxCre3MGcBJIAi65KqgK4/hWsubexcHMCJ4LepGpD3QnNMDb\n3VEppUqIzSY8dFsMnzzYgcwsO0PeXc+MHw7prbR8nL1/U9MY08elkVQ0Z34h88PBnMoM5Gn/F5ky\noReVA33cHZVSygVa1wljyR+68OSnO3lx8a9sOBjLK0ObE+KvPyzB+RrNehFp5tJIyqFC+6DHHiDz\ng4HEpnnwhM8LvPVgPyIr+ZZ+gEqpUhPi78200a15tn9j1uw5y+1v/sA2HScNcD7RdAa2isgeEdkp\nIj+LyM7rvaiIhInIChHZ53i/YmBOEekmIttzvVJFZJBj3wcicijXvpaO7SIib4rIfkecN19vjNct\n4RgZM+/gYkoaf/D6G689OJDqIX6lHoZSqvSJCOM6R/HZQ52w2WD4lA28990B7PYb+1aas4mmL1Af\n+B1WW01/Ch+SxhmTgFXGmPrAKsd6HsaY1caYlsaYlkB3IBlYnqvIU9n7jTHb88VZH5gAvFuMGIsu\n6SwZMweQmhTPo57P8q8JQ6gV5l+qISil3K9FrRAW/74LvRpX4aWluxk3azNxl9LdHZbbOJVojDFH\nCnoV47oDgVmO5VnAoGuUHwosNcYkO3He2cayEQgRkWrFiNN5yXGkzxxAZuIJfm/7M397YCTREYGl\ncmmlVNkT7OfFO/fczN8HNmHd/lj6/Xctmw7FuTsst3DX2CdVjDGnABzvkdcoPwL4KN+2fzpuj70u\nItmt7DWAY7nKHHdsu4KITBCRLSKy5dy5c0X/BLmlXSR99hCI3ccf+BN/Gn8f9asEFe+cSqlyT0QY\n1bEuXzzSCV8vGyOmbuDtb/fdcLfSXJZoRGSliOwq4DWwiOepBjQDvsm1+RmgEdAWCAOezi5ewCkK\n/Bs1xkw1xrQxxrSJiIgoSkh5eJk00ufchcfp7TxpHufR8eNpXL3SdZ9PKVXxNK0RzOLHutC/eXX+\ns3wv983cxLmLN85I0C57PN0Y07OwfSJyRkSqGWNOORLJ1WYVGg4sMMbkPHabXRsC0kRkJtbU0mDV\nYGrlOrYm1nM/LvHc+SeplnkMT3OBSfaJ3DduIs1rhrjqckqpcizQx5P/jmjJLfXCeW7hL/R7cy3/\nvaslnepVdndoLueuW2eLuDyywH1YD4QWZiT5bptlt7uINRXlIGBXrvOOdvQ+6wAk5kpKJc7XnkSY\nSeQF+zjuHPMErevorNZKqcKJCHe1rc2iRzsT7OfFPe//yGsr9pJVwW+luSvRTAZ6icg+oJdjHRFp\nIyLTswuJSF2sGsp3+Y6fKyI/Az8DlYF/OLYvAQ4C+7GmNnjEdR8BlmW1Y1T6JHqOmkSH6HBXXkop\nVYE0rBrEokdvYcjNNXlz1T7unraRMxdS3R2Wy7hlZEdjTCzQo4DtW4DxudYPU0BjvjGmeyHnNVgD\ngJaK76vdT1pmFl3qX38bj1LqxuTv7cl/hrWgY3Q4f/1yF33/u5bXhrega8Nr9Y0qf3TGrWLy8dSh\n/pVS129I65p89fvORAb5MGbmZl5etpuMLLu7wypRmmiUUsrN6kUG8uXEWxjZrjbvrjnAiKkbOZmQ\n4u6wSowmGqWUKgN8vTx46c5mvDmyFbtPXaDfm2tZ+esZd4dVIjTRKKVUGTKgRXUWP9aFGiF+jJ+9\nhX8s/pX0zPJ9K02neSxERkYGx48fJzW18J4gE1tZg2X+9ttvpRVWmeLr60vNmjXx8tKJ3JQqSVGV\nA/j84U68tOQ3pv9wiM1H4nl7ZKtyO3aiJppCHD9+nKCgIOrWrYv1uM6VvM8lARBzA45pZowhNjaW\n48ePExUV5e5wlKpwfL08eGFgUzpEh/Onz3fS7821vDK0OX2als7wjSVJb50VIjU1lfDw8EKTDFgJ\n5kZMMmA9eBYeHn7VGp9Sqvj6NqvGkse6EF05gIfmbOP5hbtIzchyd1hFoonmKq6WZApS6ERoFVRR\n/3yUUtenVpg/nz7UifGdo5i14QhD3l3P4fOX3B2W0zTRlGGBgZdrS3369CEkJIT+/fsXWHbixIm0\nbNmSxo0b4+fnR8uWLWnZsiWfffZZka65bds2li1bVqy4lVIlz9vTxl/7N2b66DYcj0+h/1s/sGiH\ny4ZyLFHaRlNOPPXUUyQnJ/Pee+8VuP9///sfAIcPH6Z///5s3769wHLXsm3bNnbt2kWfPn2uO1al\nlOv0bFyFJX/owmMf/cRjH/3EhgOxPH9HY3y9yu7D41qjKSd69OhBUND1zXGzb98+evfuTevWrbn1\n1lvZu3cvAPPnz6dp06a0aNGCbt26kZKSwosvvsjcuXOvqzaklCodNUL8mD+hAw93jeGjTUcZ9L91\n7D+b5O6wCqU1Gie88NUv/HrywjXL/XrKKuNMO03j6pV4/o4mxY7NGRMmTGD69OnExMSwbt06Hn30\nUZYvX84LL7zAmjVrqFKlCgkJCfj5+fHcc8+xa9cu3njjjVKJTSl1fbw8bDzdpxHto8J44pMd3PHW\nD/xjUFOGtK7p7tCuoImmgktISGDjxo0MGTIkZ1tmZiYAt9xyC6NHj2bYsGHceeed7gpRKVUMXRtG\nstRxK+3/Pt3BhoOxvDiwCf7eZefrvexEUoY5W/PIrsl8/GBHV4ZTJMYYKleuXGCbzbRp0/jxxx9Z\nvHgxLVq0YOfOnW6IUKnypSz9/85WpZIvc8e3581v9/PWt/vYfiyB/919Mw2rlo0p5bWNpoILDQ2l\nWrVqLFiwAAC73c6OHTsAOHjwIB06dODvf/87oaGhnDhxgqCgIC5evOjOkJVS18HTw8YTvRowZ1x7\nEpIzGPD2D8zfdBRr9hT30kRTTnTp0oVhw4axatUqatasyTfffOP0sfPnz2fKlCm0aNGCJk2asHjx\nYgAef/xxmjVrRrNmzejZsydNmzale/fu7Nixg1atWmlnAKXKoVvqVWbpH7rQtm4Yk774mT9+vJ2k\ntEy3xqS3zsqwpKTLvUjWrl3r1DF169Zl165debZFR0cXmJgWLVp0xbaIiAi2bNlSxEiVUmVJRJAP\ns+5vx7tr9vPair3sPJ7I23e3okn1YLfEozWaEvTxgx3L5P1bpdSNx8MmPNq9PvMndCQlPYvB76zn\nww2H3XIrTRONUkpVYO2iwljyhy50ignn2YW/MHHeNhJTMko1Bk00SilVwYUFeDPjvrY807cRy385\nQ/+31rLjWEKpXd8tiUZEwkRkhYjsc7yHFlCmm4hsz/VKFZFBjn1rc20/KSJfOrZ3FZHEXPueK+3P\nppRSZZHNJjx4WwwfP9gRux2GTlnPqcTUUrmV5q4azSRglTGmPrDKsZ6HMWa1MaalMaYl0B1IBpY7\n9nXJtW8D8EWuQ9dm7zPGvOjyT6KUUuVI6zqhfP1YZ7o2jORoXDJH45Jdfk13JZqBwCzH8ixg0DXK\nDwWWGmPy/ImISBBWEvqyxCO8HjNvt15KKVWGhfh7M3VUa+qE+RMR5OPy67kr0VQxxpwCcLxHXqP8\nCOCjArYPxqoZ5R6IrKOI7BCRpSJS6CP9IjJBRLaIyJZz584VNf5SUdrTBCxYsIBXXnml2HErpco+\nEaFqsG+pDFXjsiuIyEqgagG7/lLE81QDmgEFPaE4Epiea30bUMcYkyQi/bBqOvULOq8xZiowFaBN\nmzbuf3T2GkpqmoDMzEw8PQv+ax88eHDJBKuUUrm4LNEYY3oWtk9EzohINWPMKUciOXuVUw0HFhhj\n8vTHE5FwoB1WrSb7mhdyLS8RkXdEpLIx5vx1f5AyokePHqxZs+a6ju3cuTO33XYba9eu5c477yQq\nKop//etfpKenExERwZw5c4iMjGT69Ok5Izffe++9hIeHs3nzZk6fPs2rr76qiUgpdV3cNTLAIuA+\nYLLjfeFVyo4Enilg+zBgsTEmZ9J6EakKnDHGGBFph3VrMLbY0S6dBKd/vna5045BKZ1pp6naDPpO\nLl5cRXDhwgW+//57AOLj4xkwYAAiwpQpU3j11Vd5+eWXrzjm7NmzrFu3jp9//pnhw4drolFKXRd3\nJZrJwCciMg44ipU0EJE2wEPGmPGO9bpALeC7As4xwnGe3IYCD4tIJpACjDBlYUS5MmDEiBE5y0eP\nHmX48OGcPn2atLQ0GjRoUOAxgwYNQkRo3rw5J06cKK1QlVIVjFsSjTEmFuhRwPYtwPhc64eBGoWc\no2sB294G3i6pOHM4W/PIrsmM/brEQyiugICAnOWJEyfy5z//mX79+rFy5UomTy748/n4XO6Novla\nKXW9dGSAG1BiYiI1atTAGMOsWbOufYBSShWDJppyojjTBOT3t7/9jcGDB3PbbbdRpUqVEoxSKaWu\npNMElGElNU3ADz/8kGd9yJAheaZ2zjZ+fM5dS+bMmVNoLEopVRSaaEpSGWybUUopd9NbZ0oppVxK\nE41SSimX0kSjlFLKpTTRKKWUcilNNCVo7LKxjF021t1hKKVUmaKJpgzLniZg+/btdOzYkSZNmtC8\neXM+/vjjK8qWxDQBANu2bWPZsmUlEr9SSoF2by4X/P39mT17NvXr1+fkyZO0bt2a3r17ExISklPG\n2WkCrmXbtm3s2rWLPn36lEjsSimlNZpyoEGDBtSvb02rU716dSIjIynKZG379u2jd+/etG7dmltv\nvZW9e/cCMH/+fJo2bUqLFi3o1q0bKSkpvPjii8ydO/e6akNKKVUQrdE44eVNL7M7bvc1y2WXcaad\nplFYI55u93SRY9m0aRPp6enExMQ4fcyECROYPn06MTExrFu3jkcffZTly5fzwgsvsGbNGqpUqUJC\nQgJ+fn4899xzOXPSKKVUSdBEU46cOnWKUaNGMWvWLGw25yqjCQkJbNy4Mc+QM5mZmQDccsstjB49\nmmHDhnHnnXe6JGallNJE4wRnax7ZNZmZfWaWeAwXLlzg9ttv5x//+AcdOnRw+jhjDJUrVy6wzWba\ntGn8+OOPLF68mBYtWrBz586SDFkppQBtoykX0tPTGTx4cE7toyhCQ0OpVq0aCxYsAMBut7Njxw4A\nDh48SIcOHfj73/9OaGgoJ06cICgoiIsXL5b4Z1BK3bg00ZQDn3zyCd9//z0ffPBBTrflovQqmz9/\nPlOmTKFFixY0adKExV3SfuAAAAm6SURBVIsXA/D444/TrFkzmjVrRs+ePWnatCndu3dnx44dtGrV\nSjsDKKVKhN46K8Oyh+a/9957uffee506pqBpAqKjowucv2bRokVXbIuIiGDLli3XEa1SShVME00J\nckXbjFJKlXd660wppZRLuS3RiEiYiKwQkX2O99BCyv1bRH4Rkd9E5E0REcf21iLys4jsz7fdqfMq\npZQqHe6s0UwCVhlj6gOrHOt5iEgn+P/27j+2zqqO4/j7w+haMiQmUsZsCWwydIKDJYUhWzYFHGQx\n26qyzRDAKAEMxu0PI7oZAQlky4IxkSjDQRwEUWSMIT8UJ9MqYUIhwIYFM0g2OrYVK0wWYTj29Y97\nOuu4be+9vU+f3vbzSm567tNznvs9/XG/ec5z7jnMAKYCpwFnArPTt38KXAFMTo+eNVMGPK+ZmQ2d\nPBPNfGBtKq8FFhSpE0ADMBaoB+qAPZImAMdExJMREcCdvdqXcl4zMxsieSaa8RGxCyB9Pe7wChHx\nJLAJ2JUev4uIDqAJ6OxVtTMdK+m8AJKukNQuqb2cdcP6s/2SS9l+yaVVOZeZ2UiRaaKRtFHS1iKP\n+SW2PxmYAjRTSCTnSpoFqEj1KCe2iLgtIloioqWxsbGcpkNmqLcJWL9+PatWrapa/GZmkPH05og4\nv6/vSdojaUJE7EpDYV1FqrUCmyNiX2rzKHA2cBeF5NOjGXg9lUs5b02p5jYBBw4c4Mgji//aW1tb\nqx+8mY16eQ6dPQhclsqXARuK1NkBzJZ0pKQ6ChMBOtKQ2NuSzk6zzS7t1b6U89aUwW4TMHPmTJYv\nX86sWbO45ZZb2LBhA9OnT2fatGnMmTOHrq5CLl6zZg1Lly4FCh8SXbJkCeeccw6TJk06tISNmVm5\n8vzA5grgXklfo5BQLgKQ1AJcFRGXA/cB5wJbKAyN/TYifpPafx34OXAU8Gh69Hnewdh9003s7xh4\nm4B3XyrUKeU+Tf2UT3D8smVlx1LJNgFQWJSzra0NgDfffJN58+YhiVtvvZWbb76ZlStXfqBNV1cX\nTzzxBFu2bGHhwoW+4jGziuSWaCKiGzivyPF24PJUfh+4so/27RSmPJd03pGgkm0CeixevPhQeceO\nHSxcuJDdu3ezf/9+TjnllKJtFixYgCSmTp3Kzp07BxW7mY1eXoKmBKVeefRcyZx4151Vj6HSbQJ6\njBs37lD56quvZtmyZcydO5eNGzeyYsWKom3q6+sPlQuzyM3MyuclaGrAYLYJKGbv3r00NTUREaxd\nu3bgBmZmg+BEUwMGu03A4a677jpaW1uZPXs248ePr2KkZmYfJA+JQEtLSxy+NH5HRwdTpkwp6zxZ\nDp0NV5X8nMxseFi0+kkAfnXlpytqL+mZiGgZqJ7v0VTRaEowZlb7Kk0w5fLQmZmZZcqJxszMMuVE\n0w/fv+qffz5mVgonmj40NDTQ3d3tN9M+RATd3d00NDTkHYqZDXOeDNCH5uZmOjs7y1pTbLRpaGig\nubl54IpmNqo50fShrq6OiRMn5h2GmVnN89CZmZllyonGzMwy5URjZmaZ8hI0gKQ3gO0VNj8W+EcV\nw8mT+zI8jZS+jJR+gPvS48SIaByokhPNIElqL2Wtn1rgvgxPI6UvI6Uf4L6Uy0NnZmaWKScaMzPL\nlBPN4N2WdwBV5L4MTyOlLyOlH+C+lMX3aMzMLFO+ojEzs0w50VSBpBskvSDpOUmPSfpo3jFVStIq\nSS+l/qyX9OG8Y6qUpIskvSjpoKSamyEk6UJJL0vaJuk7ecdTKUl3SOqStDXvWAZL0gmSNknqSH9b\nS/KOqRKSGiQ9Jen51I/rM309D50NnqRjIuJfqfxN4JMRcVXOYVVE0hzg8Yg4IGklQERck3NYFZE0\nBTgIrAa+FRHtAzQZNiSNAf4OfA7oBJ4GvhwRf8s1sApImgXsA+6MiNPyjmcwJE0AJkTEs5I+BDwD\nLKi134skAeMiYp+kOuAvwJKI2JzF6/mKpgp6kkwyDqjZ7B0Rj0XEgfR0M1CzyzNHREdEvJx3HBU6\nC9gWEa9GxHvAL4H5OcdUkYhoA/6ZdxzVEBG7IuLZVH4b6ACa8o2qfFGwLz2tS4/M3recaKpE0o2S\nXgMuBr6fdzxV8lXg0byDGKWagNd6Pe+kBt/QRjJJJwHTgL/mG0llJI2R9BzQBfw+IjLrhxNNiSRt\nlLS1yGM+QEQsj4gTgLuBb+Qbbf8G6kuqsxw4QKE/w1YpfalRKnKsZq+URxpJRwPrgKWHjWjUjIh4\nPyLOoDBqcZakzIY1vR9NiSLi/BKr/gJ4GLg2w3AGZaC+SLoM+DxwXgzzm3hl/F5qTSdwQq/nzcDr\nOcVivaR7GuuAuyPi/rzjGayIeEvSH4ELgUwmbPiKpgokTe71dB7wUl6xDJakC4FrgHkR8e+84xnF\nngYmS5ooaSywGHgw55hGvXQT/XagIyJ+mHc8lZLU2DOjVNJRwPlk+L7lWWdVIGkd8HEKM5y2A1dF\nxM58o6qMpG1APdCdDm2u4Rl0rcCPgUbgLeC5iLgg36hKJ2ku8CNgDHBHRNyYc0gVkXQP8BkKqwTv\nAa6NiNtzDapCkmYCfwa2UPh/B1gWEY/kF1X5JE0F1lL42zoCuDcifpDZ6znRmJlZljx0ZmZmmXKi\nMTOzTDnRmJlZppxozMwsU040ZmaWKScasyEiad/Atfptf5+kSal8tKTVkl5Jq++2SZouaWwq+8PY\nNmw40ZjVAEmnAmMi4tV0aA2FhSonR8SpwFeAY9MCnH8AFuUSqFkRTjRmQ0wFq9KabFskLUrHj5D0\nk3SF8pCkRyR9KTW7GNiQ6n0MmA58LyIOAqRVnh9OdR9I9c2GBV9emw29LwBnAKdT+LT805LagBnA\nScCngOMoLEF/R2ozA7gnlU+lsMrB+32cfytwZiaRm1XAVzRmQ28mcE9aPXcP8CcKiWEm8OuIOBgR\nu4FNvdpMAN4o5eQpAb2XNuYyy50TjdnQK7YFQH/HAd4BGlL5ReB0Sf39/9YD71YQm1nVOdGYDb02\nYFHaeKoRmAU8RWE73S+mezXjKSxE2aMDOBkgIl4B2oHr02rCSJrcswePpI8Ab0TEf4aqQ2b9caIx\nG3rrgReA54HHgW+nobJ1FPah2QqsprBz497U5mH+P/FcDhwPbJO0BfgZ/9uv5rNATa0mbCObV282\nG0YkHR0R+9JVyVPAjIjYnfYM2ZSe9zUJoOcc9wPfjYiXhyBkswF51pnZ8PJQ2pBqLHBDutIhIt6R\ndC3QBOzoq3HaJO0BJxkbTnxFY2ZmmfI9GjMzy5QTjZmZZcqJxszMMuVEY2ZmmXKiMTOzTDnRmJlZ\npv4Lm9GCduUdiGgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f392d3978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    plt.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    plt.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'neg-logloss' )\n",
    "plt.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图给出了L1正则和L2正则下、不同正则参数C对应的模型在训练集上测试集上的正确率（score）。可以看出在训练集上C越大（正则越少）的模型性能越好；但在测试集上当C在1附近时性能最好，此时使用的是L1正则项\n",
    "   \n",
    "  \n",
    "如果最佳值在候选参数的边缘，最好再尝试更大的候选参数或更小的候选参数，直到找到拐点。\n",
    "这里分别是l1, c=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': array([ 0.5    ,  0.52041,  0.54082,  0.56122,  0.58163,  0.60204,\n",
       "        0.62245,  0.64286,  0.66327,  0.68367,  0.70408,  0.72449,\n",
       "        0.7449 ,  0.76531,  0.78571,  0.80612,  0.82653,  0.84694,\n",
       "        0.86735,  0.88776,  0.90816,  0.92857,  0.94898, ...673,\n",
       "        1.35714,  1.37755,  1.39796,  1.41837,  1.43878,  1.45918,\n",
       "        1.47959,  1.5    ])},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cs2 = np.linspace(0.5, 1.5)\n",
    "tuned_parameters2 = dict(penalty = penaltys, C = Cs2)\n",
    "\n",
    "grid2= GridSearchCV(lr_penalty, tuned_parameters2,cv=5, scoring='neg_log_loss')\n",
    "grid2.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.677756489458\n",
      "{'C': 0.66326530612244894, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "print(-grid2.best_score_)\n",
    "print(grid2.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最佳参数C=0.66 和penalty为L1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 用LogisticRegressionCV实现正则化的 Logistic Regression\n",
    "### 4.4.1  L1正则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegressionCV(Cs=[1, 10, 100, 1000, 10000], class_weight=None, cv=5,\n",
       "           dual=False, fit_intercept=True, intercept_scaling=1.0,\n",
       "           max_iter=100, multi_class='ovr', n_jobs=1, penalty='l1',\n",
       "           random_state=None, refit=True, scoring='neg_log_loss',\n",
       "           solver='liblinear', tol=0.0001, verbose=0)"
      ]
     },
     "execution_count": 251,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegressionCV\n",
    "\n",
    "Cs = [1, 10,100,1000, 10000]\n",
    "\n",
    "# 大量样本、高维度，L1正则 --> 可选用saga优化求解器\n",
    "# LogisticRegressionCV比GridSearchCV快\n",
    "lrcv_L1 = LogisticRegressionCV(Cs=Cs, cv = 5, scoring='neg_log_loss', penalty='l1', solver='liblinear', multi_class='ovr')\n",
    "lrcv_L1.fit(X_train, y_train)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: array([[-0.50566868, -0.53010783, -0.58890379, -0.62480124, -0.63221823],\n",
       "        [-0.50809804, -0.52561908, -0.55933841, -0.58792329, -0.61185196],\n",
       "        [-0.50430992, -0.51022066, -0.525964  , -0.5388891 , -0.54893184],\n",
       "        [-0.51907884, -0.53562397, -0.56576855, -0.584117  , -0.59713979],\n",
       "        [-0.51259991, -0.52006019, -0.53761642, -0.5543781 , -0.56397561]]),\n",
       " 1: array([[-0.47919608, -0.50235037, -0.53331912, -0.55129997, -0.56470481],\n",
       "        [-0.4937305 , -0.50951807, -0.53835704, -0.57226161, -0.60556355],\n",
       "        [-0.48698737, -0.495155  , -0.51071746, -0.52322593, -0.53939679],\n",
       "        [-0.48849348, -0.50397371, -0.52207464, -0.53670238, -0.54500009],\n",
       "        [-0.4752865 , -0.48743294, -0.50756063, -0.52192057, -0.53531371]]),\n",
       " 2: array([[-0.23764141, -0.25194796, -0.28500601, -0.30804492, -0.32557165],\n",
       "        [-0.2163503 , -0.23924135, -0.28576767, -0.31485068, -0.33093   ],\n",
       "        [-0.2258712 , -0.23145654, -0.23884582, -0.2463946 , -0.25208978],\n",
       "        [-0.23810339, -0.24385383, -0.25399498, -0.26403114, -0.27228033],\n",
       "        [-0.23702048, -0.24498208, -0.25913923, -0.27627784, -0.29359034]])}"
      ]
     },
     "execution_count": 252,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lrcv_L1.scores_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4VPW9x/H3N2GHsO+BsCPKIuAQ\ncFdcatWKVtw3XEpdqFWLvXpbeyvWe63tVaxiFRX33euCWKUqizsQkC0gEMIW9jUQIJDle/+YiTfN\nDcwAmZxJ8nk9D8+TM/M7kw8HJp+cM+f8jrk7IiIiB5MUdAAREUl8KgsREYlKZSEiIlGpLEREJCqV\nhYiIRKWyEBGRqFQWIiISlcpCRESiUlmIiEhUtYIOUFFatmzpnTt3DjqGiEiVMnv27C3u3irauGpT\nFp07dyYjIyPoGCIiVYqZrYplnA5DiYhIVCoLERGJSmUhIiJRqSxERCQqlYWIiESlshARkahUFiIi\nEpXKQkSkiiooKub1mav5ZOGGuH+vuJaFmZ1jZkvMLMvM7jnIuOFm5mYWiizXNrMXzWyBmS02s3vj\nmVNEpCopLCrmrYw1DP3vadz77gL+sWB93L9n3K7gNrNkYBxwFpADzDKzie6+qMy4FOB2YEaphy8B\n6rp7XzNrACwys9fdfWW88oqIJLqiYmfivLU89tkyVm7dQ78OTRgzrA+n9Yw6W8cRi+d0H+lAlrtn\nA5jZG8AwYFGZcQ8ADwOjSz3mQEMzqwXUB/YDO+OYVUQkYRUXO5MWrOexz5ayfPNujm7XmGeuDXHm\n0a0xs0rJEM+ySAXWlFrOAQaXHmBmA4CO7j7JzEqXxTuEi2U90AC40923xTGriEjCKS52PsncwNjP\nlrJ0Yx5HtUnhqasHcvYxbUlKqpySKBHPsijvb+I/PmmWBDwKjChnXDpQBLQHmgFfmtlnJXsppV5j\nJDASIC0trWJSi4gEzN35dNFGHv1sGYvX76Rbq4Y8fsUAzuvbrtJLokQ8yyIH6FhquQOwrtRyCtAH\nmBbZjWoLTDSzC4ArgU/cvQDYZGZfAyHgX8rC3ccD4wFCoZAjIlKFuTtTl2zi0U+XsWBtLl1aNmTs\nZf352bHtSQ6oJErEsyxmAT3MrAuwFriccAkA4O65QMuSZTObBox29wwzOwMYamavED4MNQQYG8es\nIiKBcXe+XLaFRz5dytw1O+jYvD5/Gd6PiwakUis5Ma5wiFtZuHuhmY0CJgPJwAR3zzSzMUCGu088\nyOrjgOeBhYQPZz3v7vPjlVVEJCjfZIVLImPVdlKb1uehn/fl4uM6UDtBSqKEuVePozehUMh18yMR\nqSpmrtjGI58u4bvsbbRtXI9RQ7tzaagjdWpVbkmY2Wx3D0UbV23ulCciUhXMXrWdRz9dyldZW2iV\nUpc//uwYLk9Po17t5KCjHZTKQkSkEsxds4NHP13K9KWbadmoDr8/72iuHtIp4UuihMpCRCSOFq7N\nZexnS/ls8SaaNajNPT/txbXHd6JBnar147dqpRURqSIWr9/J2M+WMjlzI03q1+bunxzFdSd0plHd\nqvljt2qmFhFJUMs27mLsZ8v4aMF6UurW4o4ze3DDSV1oXK920NGOiMpCRKQCLN+cx98+X8bEeeto\nUDuZXw3tzk0ndaVJg6pdEiVUFiIiR2DV1t387fMs3vs+h7q1kvnlKd0YeUpXmjesE3S0CqWyEBE5\nDGu27eGJKVm8MyeHWknGjSd14ZendqNlo7pBR4sLlYWIyCFYt2MvT0zN4q1Za0hKMq49vhO3nNqN\n1o3rBR0trlQWIiIx2LgznyenZvH6zDU4zhXpadx2enfaNqneJVFCZSEichCbduXz1LRsXp2xiqJi\n55JQR0YN7U5q0/pBR6tUKgsRkXJszdvH019k89K3Kykocn4+IJVfDe1BWosGQUcLhMpCRKSU7bv3\n88yX2bzwzUryC4q4sH8qvzqjB11aNgw6WqBUFiIiQO7eAp77MpsJX69k9/5Czu/Xnl+f0YPurRsF\nHS0hqCxEpEbblV/A81+v5Jkvs9mVX8i5fdvy6zN6clTblKCjJRSVhYjUSLv3FfLCN+GS2LGngLOP\nacMdZ/bkmPaNg46WkFQWIlKj7N1fxMvfreSp6dls272fob1ac+eZPenboUnQ0RKaykJEaoT8giJe\nnbGav09bzpa8fZzSsxV3ntmDAWnNgo5WJcS1LMzsHOAxwvfgftbdHzrAuOHA28Agd88ws6uAu0sN\n6QcMdPe58cwrItXPvsIi3pi5hnFTs9i0ax8ndGvB368eyKDOzYOOVqXErSzMLBkYB5wF5ACzzGyi\nuy8qMy4FuB2YUfKYu78KvBp5vi/wgYpCRA7F/sJi3p69hiemZLE+N5/0zs157PIBHN+tRdDRqqR4\n7lmkA1nung1gZm8Aw4BFZcY9ADwMjD7A61wBvB6vkCJSvRQUFfPunBz+9nkWa3fsZWBaU/56ybGc\n0K0FZhZ0vCornmWRCqwptZwDDC49wMwGAB3dfZKZHagsLiNcMiIiB1RYVMwHc9fxtynLWLV1D8d2\naMKDF/Xh1J6tVBIVIJ5lUd6/jv/4pFkS8Cgw4oAvYDYY2OPuCw/w/EhgJEBaWtqRZBWRKqqo2Jk0\nfx2PfbaM7C276d2+Mc9dF2Jor9YqiQoUz7LIATqWWu4ArCu1nAL0AaZF/kHbAhPN7AJ3z4iMuZyD\nHIJy9/HAeIBQKOQHGici1U9xsfPxwg2M/Wwpyzbl0attCk9dfRw/6d1GJREH8SyLWUAPM+sCrCX8\ng//KkifdPRdoWbJsZtOA0SVFEdnzuAQ4JY4ZRaSKcXcmZ25k7GdL+WHDLrq3bsS4Kwfy0z5tSUpS\nScRL3MrC3QvNbBQwmfCpsxPcPdPMxgAZ7j4xykucAuSUfEAuIjWbuzPlh0088ulSMtftpGvLhjx2\neX/O79eeZJVE3Jl79Th6EwqFPCMjI/pAEalS3J3pSzfz6KdLmZeTS1rzBvz6jB4M69+eWslJQcer\n8sxstruHoo3TFdwikpDcna+ztvLIp0uYs3oHqU3r8/DF/bhoYCq1VRKVTmUhIgnnu+ytPPLpUmau\n2Ea7JvV48KI+XHJcR+rUUkkERWUhIglj7Y693PvuAr5YupnWKXUZM6w3lw3qSN1ayUFHq/FUFiIS\nOHfng7nruO+DhRQXO78/72iuHtKJerVVEolCZSEigdqxZz+/e38hH81fT6hTMx65tH+Nvc91IlNZ\niEhgvlq2hd+8PZetefu5+ydHcfOp3XQabIJSWYhIpcsvKOLPn/zA81+vpHvrRjx33SD6pOrmQ4lM\nZSEilWrh2lzufHMuyzblMeKEztzz0176bKIKUFmISKUoKnaemr6csZ8tpXnDOrx8Yzon92gVdCyJ\nkcpCROJuzbY93PnmXDJWbee8fu148MI+NG1QJ+hYcghUFiISN+7O27NzuH9iJklJxtjL+jOsf3vN\nClsFqSxEJC625u3j399bwOTMjQzp2pz/vrQ/qU3rBx1LDpPKQkQq3NQfNnH3O/PZubeA3517NDee\n1EXTh1dxKgsRqTB79hfy4EeLeXXGanq1TeHlG9M5ul3joGNJBVBZiEiFmLtmB3e+OZeVW3cz8pSu\n3HVWT50SW42oLETkiBQWFfPE1Cwen5JFm5S6vHbTEI7v1iLoWFLBVBYicthWbNnNHW/OZd6aHVw0\nIJU/XtCbJvVrBx1L4kBlISKHzN15beZq/jRpMXVqJfHElQM4v1/7oGNJHMX1TiJmdo6ZLTGzLDO7\n5yDjhpuZm1mo1GP9zOxbM8s0swVmVi+eWUUkNpt25XPjixn87r2FhDo3Y/Idp6goaoC47VmYWTIw\nDjgLyAFmmdlEd19UZlwKcDswo9RjtYBXgGvcfZ6ZtQAK4pVVRGIzOXMD9767gN37Cvnjz47h2uM7\n65TYGiKeh6HSgSx3zwYwszeAYcCiMuMeAB4GRpd67GxgvrvPA3D3rXHMKSJR5O0rZMyHmbyVkUPv\n9o0Ze1l/erRJCTqWVKJ4HoZKBdaUWs6JPPYjMxsAdHT3SWXW7Qm4mU02szlm9ts45hSRg8hYuY2f\nPvYF78zO4bbTu/HerSeqKGqgeO5ZlLdv6j8+aZYEPAqMKGdcLeAkYBCwB/jczGa7++f/8g3MRgIj\nAdLS0iomtYgAsL+wmMc+X8rfpy0ntVl93vrl8YQ6Nw86lgQknmWRA3QstdwBWFdqOQXoA0yLTCrW\nFphoZhdE1p3u7lsAzOwfwEDgX8rC3ccD4wFCoZAjIhUia9Mu7nhzLgvX7uTSUAf+8LPeNKqrkydr\nsnj+688CephZF2AtcDlwZcmT7p4LtCxZNrNpwGh3zzCz5cBvzawBsB84lfBeiIjEUXGx8+K3K3no\n4x9oWLcWT19zHD/p3TboWJIA4lYW7l5oZqOAyUAyMMHdM81sDJDh7hMPsu52M3uEcOE48A93/yhe\nWUUENuTmc/c78/hy2RaG9mrNQxf3pXWKzliXMHOvHkdvQqGQZ2RkBB1DpEqaNH8dv3tvIfsLi/n9\n+UdzZXqa7jlRQ0Q+Dw5FG6eDkCI1WO7eAv44MZP3vl/LsR2b8uilx9K1VaOgY0kCUlmI1FDfLt/K\nb96ay8Zd+7jjzB6MOr07tZLjOqmDVGEqC5EaZl9hEX+dvIRnv1pB5xYN+Z9bTqB/x6ZBx5IEp7IQ\nqUEWr9/JnW/O5YcNu7hqcBq/O+9oGtTRjwGJTv9LRGqA4mLn2a+y+evkpTSuX5sJI0IM7dUm6FhS\nhagsRKq5tTv28pu35vJd9jbOPqYN//XzvrRoVDfoWFLFqCxEqil354O567jvg4UUFzsPD+/HJcd1\n0CmxclhUFiLV0I49+/nd+wv5aP56Qp2a8cil/Ulr0SDoWFKFqSxEqpkvl21m9Nvz2Jq3n7t/chQ3\nn9qNZN1zQo6QykKkmsgvKOKhj3/ghW9W0r11I567bhB9UpsEHUuqiZjKwsxOBOa6+24zu5rwDLCP\nufuquKYTkZgsXJvLHW/OJWtTHiNO6Mw9P+1FvdrJQceSaiTWyzX/Duwxs2OB3wKrgJfilkpEYlJU\n7IybmsWF475mV34BL9+Yzh8v6K2ikAoX62GoQnd3MxtGeI/iOTO7Lp7BROTgVm/dw11vzSVj1XbO\n69eOBy/sQ9MGdYKOJdVUrGWxy8zuBa4GTjGzZKB2/GKJyIG4O29n5HD/h5kkJRljL+vPsP7tdUqs\nxFWsZXEZ4RsX3ejuG8wsDfhL/GKJSHm25u3j3ncX8M9FGxnStTn/fWl/UpvWDzqW1AAx71kQPvxU\nZGY9gV7A6/GLJSJlTflhI799ZwE79xbwu3OP5saTupCkU2KlksRaFl8AJ5tZM8L3wc4gvLdxVbyC\niUjYnv2FPPjRYl6dsZpebVN45aZ0erVtHHQsqWFiLQtz9z1mdiPwuLs/bGZz4xlMROD71du56615\nrNy6m5GndOU3Z/ekbi2d6SSVL+ayMLPjCe9J3Bh5TP9jReKkoKiYJ6Zk8cTULNo2rsdrNw3h+G4t\ngo4lNVis11ncAdwLvOfumWbWFZgabSUzO8fMlphZlpndc5Bxw83MzSwUWe5sZnvNbG7kz1Mx5hSp\n8rI35zH8qW957PNlDDu2PR/fcbKKQgIX056Fu08HpptZipk1cvds4PaDrRM5vXYccBaQA8wys4nu\nvqjMuJTIa80o8xLL3b1/jH8PkSrP3Xl1xmoe/GgxdWolMe7KgZzXr13QsUSA2Kf76Ev4iu3m4UXb\nDFzr7pkHWS0dyIoUC2b2BjAMWFRm3APAw8DoQ8wuUm1s2pXPv70zn6lLNnNyj5b8ZfixtG1SL+hY\nIj+K9TDU08Bd7t7J3dOA3wDPRFknFVhTajkn8tiPzGwA0NHdJ5Wzfhcz+97MppvZyeV9AzMbaWYZ\nZpaxefPmGP8qIollcuYGzhn7Jd8s38off3YML16frqKQhBPrB9wN3f3HzyjcfZqZNYyyTnkngPuP\nT5olAY8CI8oZtx5Ic/etZnYc8L6Z9Xb3nf/yYu7jgfEAoVDIy3kdkYSVt6+Q+ydm8vbsHHq3b8zY\ny/rTo01K0LFEyhVrWWSb2X3Ay5Hlq4EVUdbJATqWWu4ArCu1nAL0AaZFpiloC0w0swvcPQPYB+Du\ns81sOdCT8PUdIlXerJXbuOutuazdvpfbTu/Gr8/oSZ1ase7oi1S+WMviBuB+4F3CewxfANdHWWcW\n0MPMugBrgcsJTxkCgLvnAi1Lls1sGjDa3TPMrBWwLXLFeFegB5AdY1aRhLW/sJixny3lqenLSW1W\nn7d+eTyhzs2DjiUSVaxnQ20nytlP5axTaGajgMmEr8mYEDntdgyQ4e4TD7L6KcAYMysEioCb3X3b\noXx/kUSzbOMu7nhzLpnrdnJpqAN/+FlvGtXV/cekajD3Ax/qN7MPKfU5Q1nufkE8Qh2OUCjkGRk6\nSiWJp7jYefHblTz08Q80rFuL//p5X37Su23QsUQAMLPZ7h6KNi7arzV/raA8IjXSzvwCRr32PV8s\n3czQXq156OK+tE7RmU5S9Ry0LCIX44nIYdi0M5/rnp9F1qZdPHBhH64enKZ7TkiVFetFeQv4/4ej\ncgmfnfQnd99a0cFEqrIVW3Zz7YQZbM3bz3PXDeKUnq2CjiRyRGL9dO1jwh80vxZZvpzwWVG5wAvA\nzyo8mUgVtSAnlxHPz8SBN0YOoV+HpkFHEjlisZbFie5+YqnlBWb2tbufaGZXxyOYSFX05bLN3Pzy\nbJo1rMNLN6TTtVWjoCOJVIhYrwJqZGaDSxbMLB0oeRcUVngqkSpo4rx13PDCLDo2b8D/3HKCikKq\nlVj3LG4CJphZI8KHn3YCN0am/PiveIUTqSqe/3oF93+4iPQuzXnm2hBN6tcOOpJIhYr1orxZQF8z\na0L42owdpZ5+Ky7JRKoAd+ev/1zCuKnL+UnvNjx2+QDq1dZ9waT6ifVsqCbAfxC+shozmw6MiUzZ\nIVIjFRYV87v3FvJmxhquSE/jTxf2ITlJp8ZK9RTrZxYTgF3ApZE/O4Hn4xVKJNHlFxRx8ytzeDNj\nDbcP7c5/XqSikOot1s8surn7xaWW7zezufEIJJLocvcUcNNLs8hYtZ0xw3pz7fGdg44kEnex7lns\nNbOTShbM7ERgb3wiiSSuDbn5XPr0t8xbk8sTVwxUUUiNEeuexS3AiyUfcAPbKP+mRSLVVtamPK6b\nMJPcvQW8cP0gTujeMvpKItVErGdDzQWONbPGkeWdUVYRqVbmrtnB9c/PJDkpiTdGDqFPapOgI4lU\nqoOWhZnddYDHAXD3R+KQSSShTFuyiVtemUOrlLq8fGM6nVpEu6OwSPUTbc9CNwSWGu2973O4++35\nHNU2hReuT6dVSt2gI4kEItoU5fdXVhCRRPPsl9n86aPFnNCtBU9fcxwp9XRVttRch3xPRzOb4+4D\n4xFGJBG4Ow99/ANPf5HNuX3b8uhl/albS1dlS80W66mzpcV85ZGZnWNmS8wsy8zuOci44WbmZhYq\n83iameWZ2ejDyClyyAqKihn99nye/iKba4Z04vErBqooRDiMPQvgo1gGmVkyMA44C8gBZpnZRHdf\nVGZcCnA7MKOcl3mU8L00ROJuz/5Cbnt1DlOXbOaus3ryq6HddWc7kYhD3rNw99/HODQdyHL3bHff\nD7wBDCtn3APAw0B+6QfN7EIgG8g81Iwih2r77v1c9ewMpi/dzH9e1Jfbz+ihohApJaayMLNdZraz\nzJ81ZvaemXU9wGqpwJpSyzmRx0q/7gCgo7tPKvN4Q+DfgIN+wG5mI80sw8wyNm/eHMtfReT/Wbdj\nL5c8/S2Z63by5FUDuXJwWtCRRBJOrIehHgHWEb6tqhG+rWpbYAnhSQZPK2ed8n4t+/E+3maWRPgw\n04hyxt0PPOrueQf77c7dxwPjAUKhUNl7hItEtWzjLq6dMJO8/EJeuiGdIV1bBB1JJCHFWhbnuPvg\nUsvjzew7dx9jZv9+gHVygI6lljsQLpwSKUAfYFqkENoCE83sAmAwMNzMHgaaAsVmlu/uT8SYVySq\n2au2c8MLs6hTK4k3f3k8x7RvHHQkkYQVa1kUm9mlwDuR5eGlnjvQb/SzgB5m1gVYS3hv5MofVwrf\nC+PHyXXMbBow2t0zgJNLPf5HIE9FIRVpyg8bufXVObRrUp+XbkinY/MGQUcSSWixfsB9FXANsAnY\nGPn6ajOrD4wqbwV3L4w8NxlYDLzl7plmNiay9yASiLcz1vCLl2bTs00K79x8vIpCJAbmXj0O9YdC\nIc/IyAg6hiQwd+ep6dn8+ZMfOLlHS/5+9XE0qns4Z4+LVB9mNtvdQ9HGxXo2VE8z+9zMFkaW+5lZ\nrKfQigSuuNj500eL+fMnP3DBse157rpBKgqRQxDrYahngHuBAgB3n0/4MwiRhLe/sJg735rLc1+t\nYMQJnRl7WX/q1DqcyQtEaq5Yf7Vq4O4zy5zGWhiHPCIVave+Qm55dQ5fLN3Mb885iltO7aaL7UQO\nQ6xlscXMuhE588nMhgPr45ZKpAJszdvHDS/MYsHaXB6+uB+XDuoYfSURKVesZXEb4YvfepnZWmAF\n4TOkRBJSzvY9XPvcTNbu2MvT14Q465g2QUcSqdJiLYu1wPPAVKA5sBO4DhgTp1wih+2HDTu5bsJM\n9u4v4pWbBjOoc/OgI4lUebGWxQfADmAO/3oVtkhCmbliGze+OIsGdZJ5++YTOKqtbvYoUhFiLYsO\n7n5OXJOIHKF/Zm7gV69/T2qz8FXZHZrpYjuRihLr+YPfmFnfuCYROQJvzFzNza/M5uh2jXnn5hNU\nFCIVLNY9i5OAEWa2AthHeEZZd/d+cUsmEgN3Z9zULP76z6WcdlQrnrxqIA3q6GI7kYoW67vqp3FN\nIXIYioud+z/M5MVvV3HRgFQeHt6P2sm62E4kHmIqC3dfFe8gIodiX2ERd701j4/mr+cXJ3fh3p8e\nTVKSLrYTiRftr0uVk7evkF++nMHXWVv593N7MfKUbkFHEqn2VBZSpWzJ28eI52eyeP0u/vuSY7n4\nuA5BRxKpEVQWUmWs3rqHayfMYMPOfJ69NsTpvVoHHUmkxlBZSJWQuS6XEc/PoqComNd+MYSBac2C\njiRSo6gsJOF9u3wrI1/KIKVeLV7/xfF0b62rskUqm8pCEtrHC9bz6zfm0qlFA168IZ32TesHHUmk\nRorrSelmdo6ZLTGzLDO75yDjhpuZm1kospxuZnMjf+aZ2UXxzCmJ6ZXvVnHra3Po26EJb998vIpC\nJEBx27Mws2RgHHAWkAPMMrOJ7r6ozLgU4HZgRqmHFwIhdy80s3bAPDP70N11w6UawN157PNljP1s\nGWf0as0TVw6kfp3koGOJ1Gjx3LNIB7LcPdvd9wNvAMPKGfcA8DCQX/KAu+8pVQz1iNx0Saq/omLn\n9+8vZOxny7jkuA48fc1xKgqRBBDPskgF1pRazok89iMzGwB0dPdJZVc2s8FmlgksAG7WXkX1l19Q\nxG2vzuHVGau55bRuPDy8H7U0fYdIQojnB9zlzb3w4x6CmSUBjwIjylvZ3WcAvc3saOBFM/vY3fNL\njzGzkcBIgLS0tAqKLUHYmV/AyJcy+C57G/edfww3ntQl6EgiUko8f23LAUrf9LgD/3rjpBSgDzDN\nzFYCQ4CJJR9yl3D3xcDuyFjKPDfe3UPuHmrVqlUFx5fKsmlnPpc9/R0ZK7fz2OX9VRQiCSieexaz\ngB5m1oXwbVkvB64sedLdc4GWJctmNg0Y7e4ZkXXWRD7g7gQcBayMY1YJyMotu7lmwgy25u1nwohB\nnNJTpS+SiOJWFpEf9KOAyUAyMMHdM81sDJDh7hMPsvpJwD1mVgAUA7e6+5Z4ZZVgLMjJZcTzM3Hg\ntV8MoX/HpkFHEpEDMPfqcaJRKBTyjIyMoGNIjL5atoVfvpxB0wZ1eOnGdLq1ahR0JJEaycxmu3so\n2jhdwS2VbtL8ddz55ly6tWrEizek06ZxvaAjiUgUKgupVC9+s5I/fpjJoE7Neea6EE3q1w46kojE\nQGUhlcLdeeTTpTw+JYuzj2nD364YQL3authOpKpQWUjcFRYV8/v3F/LGrDVckd6RB4b10cV2IlWM\nykLiKr+giF+9/j2fLtrI7UO7c+dZPTHTvbJFqhqVhcRN7p4CbnppFhmrtnP/Bb257oTOQUcSkcOk\nspC42Lgzn2ufm0n2ljwev2IA5/drH3QkETkCKgupcMs353HtczPZsWc/L1yfzondW0ZfSUQSmspC\nKtTcNTu4/vmZJCcZb/7yePqkNgk6kohUAJWFVJjpSzdzyyuzadmoLi/dkE7nlg2DjiQiFURlIRXi\n/e/XMvrtefRok8KLNwyidYquyhapTlQWcsSe+2oFD0xaxJCuzRl/bYjG9XRVtkh1o7KQw+bu/PmT\nJTw1fTnn9m3LI5f211XZItWUykIOS2FRMfe8u4B3Zudw9ZA07r+gD8lJuthOpLpSWcgh27u/iFGv\nzeHzHzZx55k9uf2M7roqW6SaU1nIIZmfs4M/fJDJ/JwdPHhRH64a3CnoSCJSCVQWEpNZK7fx+JQs\nvli6mcb1avHkVQM5p0+7oGOJSCVRWcgBuTtfZ23l8SnLmLFiGy0a1uHfzunF1UPSSNEZTyI1SlzL\nwszOAR4jfA/uZ939oQOMGw68DQxy9wwzOwt4CKgD7Afudvcp8cwq/8fdmfLDJh6fksXcNTto07gu\nfzj/GK5IT6N+HZ3tJFITxa0szCwZGAecBeQAs8xsorsvKjMuBbgdmFHq4S3Az9x9nZn1ASYDqfHK\nKmHFxc4nmRt4fEoWi9fvpEOz+jx4UR+GH9eBurVUEiI1WTz3LNKBLHfPBjCzN4BhwKIy4x4AHgZG\nlzzg7t+Xej4TqGdmdd19Xxzz1liFRcVMnLeOJ6ctJ2tTHl1bNuSvlxzLsP7tqa2bFIkI8S2LVGBN\nqeUcYHDpAWY2AOjo7pPMbDTluxj4XkVR8fYVFvHunLX8fdpyVm/bQ6+2KTx+xQDO7dtO10yIyL+I\nZ1mU99PGf3zSLAl4FBhxwBcw6w38GTj7AM+PBEYCpKWlHUHUmiW/oIg3Zq7m6S+yWZ+bT78OTbjv\n/BBn9GpNkkpCRMoRz7LIATpvP0dcAAAMG0lEQVSWWu4ArCu1nAL0AaZFLuhqC0w0swsiH3J3AN4D\nrnX35eV9A3cfD4wHCIVCXt4Y+T95+wp59btVPPPlCrbk7WNQ52b8+eJ+nNyjpS6qE5GDimdZzAJ6\nmFkXYC1wOXBlyZPungv8eFccM5sGjI4URVPgI+Bed/86jhlrhNy9Bbz4zUomfL2CHXsKOLlHS0ad\nPoDBXVsEHU1Eqoi4lYW7F5rZKMJnMiUDE9w908zGABnuPvEgq48CugP3mdl9kcfOdvdN8cpbHW3N\n28dzX63gpW9XkbevkDOPbs1tp3dnQFqzoKOJSBVj7tXj6E0oFPKMjIygYySEjTvzGf9FNq/NWE1+\nYRHn9m3Hbad155j2jYOOJiIJxsxmu3so2jhdwV2N5Gzfw1PTl/PWrByK3Bl2bHtuPb073Vs3Cjqa\niFRxKotqYMWW3Tw5NYv3vl+LGQw/riO3nNqNtBYNgo4mItWEyqIKW7JhF+OmZjFp/jpqJydx9ZBO\njDylK+2b1g86mohUMyqLKmhBTi5PTF3G5MyNNKyTzC9O6cpNJ3WlVUrdoKOJSDWlsqhCMiLThE+P\nTBN++xk9uP6EzjRrWCfoaCJSzaksEpy7883y8DTh32Vvo3nDOtz9k6O45vhONNY04SJSSVQWCcrd\nmbokPE3496t30DqlLvedfwxXpHekQR39s4lI5dJPnQRTXOxMjkwTvmj9TlKb1udPF4anCa9XW9OE\ni0gwVBYJorComA/nr2Pc1PA04V1aNuQvw/tx4YBUTRMuIoFTWQRsf2Ex732fw5PTlrNq6x6OapPC\n364YwHmaJlxEEojKIiD5BUW8OWsNT09fzrrcfPqmNuHpa47jrKPbaJpwEUk4KotKtntfIa/OWMX4\nL8LThIc6NeM/f96XU3u20jThIpKwVBaVJHdvAS99s5LnItOEn9S9JaOGDmBwl+YqCRFJeCqLONu2\nez8TvlrBi9+sZNe+Qs7o1ZrbhnZnoKYJF5EqRGURJ5t25vPMl9m88l14mvCf9mnLrad1p09qk6Cj\niYgcMpVFBcvZvoenp2fzZsYaCouKGdY/lVtP60aPNilBRxMROWwqiwqyYstu/j4ti3fnlEwT3oGb\nT+1GpxYNg44mInLEVBZHaOnG8DThH87TNOEiUn3FtSzM7BzgMcL34H7W3R86wLjhwNvAIHfPMLMW\nwDvAIOAFdx8Vz5yHY+HaXJ6YksUnmRtoUCeZX5zclRtP7kLrlHpBRxMRqXBxKwszSwbGAWcBOcAs\nM5vo7ovKjEsBbgdmlHo4H7gP6BP5kzBmr9rOE1OWMXXJZlLq1eL2od25/sQumiZcRKq1eO5ZpANZ\n7p4NYGZvAMOARWXGPQA8DIwuecDddwNfmVn3OOaLmbvzbfZWnpiSxTfLt9KsQW1NEy4iNUo8yyIV\nWFNqOQcYXHqAmQ0AOrr7JDMbTYJxd6Yt2czjU5YxZ/UOWqXU5ffnHc2Vg9M0TbiI1Cjx/IlX3mXJ\n/uOTZknAo8CIw/4GZiOBkQBpaWmH+zL/T3Gx889F4WnCM9eFpwl/YFhvLgl11DThIlIjxbMscoCO\npZY7AOtKLacQ/jxiWmS6i7bARDO7wN0zYvkG7j4eGA8QCoU8yvCoCouK+WjBep6YksWyTXl0btGA\nh4f348L+qdSppWnCRaTmimdZzAJ6mFkXYC1wOXBlyZPungu0LFk2s2nA6FiLoiLtLyzm/e/X8uS0\nLFZu3UPPNo147PL+nNe3HbV0LwkRkfiVhbsXmtkoYDLhU2cnuHummY0BMtx94sHWN7OVQGOgjpld\nCJxd9kyqijBvzQ5ufXUOa3fspU9qY566+jjOPkbThIuIlBbXT2nd/R/AP8o89ocDjD2tzHLnuAUr\npXOLhnRt1ZA/XdSH0zRNuIhIuWr8KT1NGtTm5RsHRx8oIlKD6YC8iIhEpbIQEZGoVBYiIhKVykJE\nRKJSWYiISFQqCxERiUplISIiUaksREQkKnM/4vn3EoKZbQZWHcFLtAS2VFCciqRch0a5Do1yHZrq\nmKuTu7eKNqjalMWRMrMMdw8FnaMs5To0ynVolOvQ1ORcOgwlIiJRqSxERCQqlcX/GR90gANQrkOj\nXIdGuQ5Njc2lzyxERCQq7VmIiEhUNaoszOwcM1tiZllmdk85z9c1szcjz88ws84JkmuEmW02s7mR\nPzdVUq4JZrbJzBYe4Hkzs79Fcs83s4EJkus0M8sttb3KveFWHHJ1NLOpZrbYzDLN7NfljKn0bRZj\nrkrfZmZWz8xmmtm8SK77yxlT6e/JGHMF9Z5MNrPvzWxSOc/Fd1u5e434Q/jWrsuBrkAdYB5wTJkx\ntwJPRb6+HHgzQXKNAJ4IYJudAgwEFh7g+XOBjwEDhgAzEiTXacCkALZXO2Bg5OsUYGk5/5aVvs1i\nzFXp2yyyDRpFvq4NzACGlBkTxHsyllxBvSfvAl4r798q3tuqJu1ZpANZ7p7t7vuBN4BhZcYMA16M\nfP0OcIbF/z6rseQKhLt/AWw7yJBhwEse9h3Q1MzaJUCuQLj7enefE/l6F7AYSC0zrNK3WYy5Kl1k\nG+RFFmtH/pT9ELXS35Mx5qp0ZtYBOA949gBD4rqtalJZpAJrSi3n8P/fMD+OcfdCIBdokQC5AC6O\nHLZ4x8w6xjlTrGLNHoTjI4cRPjaz3pX9zSOHAAYQ/q20tEC32UFyQQDbLHJYZS6wCfjU3Q+4vSrx\nPRlLLqj89+RY4LdA8QGej+u2qkllUV7Dlv1tIZYxFS2W7/kh0Nnd+wGf8X+/PQQtiO0VizmEpzA4\nFngceL8yv7mZNQL+B7jD3XeWfbqcVSplm0XJFcg2c/cid+8PdADSzaxPmSGBbK8YclXqe9LMzgc2\nufvsgw0r57EK21Y1qSxygNLt3wFYd6AxZlYLaEL8D3dEzeXuW919X2TxGeC4OGeKVSzbtNK5+86S\nwwju/g+gtpm1rIzvbWa1Cf9AftXd3y1nSCDbLFquILdZ5HvuAKYB55R5Koj3ZNRcAbwnTwQuMLOV\nhA9VDzWzV8qMieu2qkllMQvoYWZdzKwO4Q+AJpYZMxG4LvL1cGCKRz4tCjJXmWPaFxA+5pwIJgLX\nRs7wGQLkuvv6oEOZWduSY7Vmlk74//nWSvi+BjwHLHb3Rw4wrNK3WSy5gthmZtbKzJpGvq4PnAn8\nUGZYpb8nY8lV2e9Jd7/X3Tu4e2fCPyOmuPvVZYbFdVvVqqgXSnTuXmhmo4DJhM9AmuDumWY2Bshw\n94mE31Avm1kW4Ua+PEFy3W5mFwCFkVwj4p0LwMxeJ3yWTEszywH+g/CHfbj7U8A/CJ/dkwXsAa5P\nkFzDgVvMrBDYC1xeCaUP4d/+rgEWRI53A/w7kFYqWxDbLJZcQWyzdsCLZpZMuJzecvdJQb8nY8wV\nyHuyrMrcVrqCW0REoqpJh6FEROQwqSxERCQqlYWIiESlshARkahUFiIiEpXKQuQQmFle9FEHXf8d\nM+sa+bqRmT1tZssjs5t+YWaDzaxO5Osac2q7JD6VhUglicy3lOzu2ZGHniV8PnwPd+9N+Fz9lpEJ\nJT8HLgskqEg5VBYihyFyBfZfzGyhmS0ws8sijyeZ2ZORPYVJZvYPMxseWe0q4IPIuG7AYOD37l4M\nEJl5+KPI2Pcj40USgnZzRQ7Pz4H+wLFAS2CWmX1B+GrpzkBfoDXhaSAmRNY5EXg98nVvYK67Fx3g\n9RcCg+KSXOQwaM9C5PCcBLwemZ10IzCd8A/3k4C33b3Y3TcAU0ut0w7YHMuLR0pkv5mlVHBukcOi\nshA5PAe6qczBbjazF6gX+ToTONbMDvYerAvkH0Y2kQqnshA5PF8Al0VuktOK8K1eZwJfEb4pTpKZ\ntSE84WGJxUB3AHdfDmQA95ea7bWHmQ2LfN0C2OzuBZX1FxI5GJWFyOF5D5hP+J7pU4DfRg47/Q/h\n+wosBJ4mfEe63Mg6H/Gv5XET0BbIMrMFhO+LUHJvi9MJz1ArkhA066xIBTOzRu6eF9k7mAmc6O4b\nIvdGmBpZPtAH2yWv8S5wr7svqYTIIlHpbCiRijcpcvOcOsADkT0O3H2vmf0H4Xslrz7QypGbYL2v\nopBEoj0LERGJSp9ZiIhIVCoLERGJSmUhIiJRqSxERCQqlYWIiESlshARkaj+F5ecCo5xuwbSAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f398d7208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C is: [  1.   10.    0.1]\n"
     ]
    }
   ],
   "source": [
    "# scores_：dict with classes as the keys, and the values as the grid of scores obtained during cross-validating each fold,\n",
    "# Each dict value has shape (n_folds, len(Cs))\n",
    "n_Cs = len(Cs)\n",
    "n_classes = 3\n",
    "scores =  np.zeros((n_classes,n_Cs))\n",
    "\n",
    "for j in range(n_classes):\n",
    "        scores[j][:] = np.mean(lrcv_L1.scores_[j],axis = 0)\n",
    "    \n",
    "mse_mean = -np.mean(scores, axis = 0)\n",
    "plt.plot(np.log10(Cs), mse_mean.reshape(n_Cs,1)) \n",
    "#plt.plot(np.log10(reg.Cs)*np.ones(3), [0.28, 0.29, 0.30])\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('neg-logloss')\n",
    "plt.show()\n",
    "print ('C is:',lr_cv_L2.C_)  #对多类分类问题，每个类别的分类器有一个C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ -9.20913257e-02,  -3.34858143e-01,  -4.93084248e-03,\n",
       "          2.60631288e-03,   8.28259624e-04,  -1.61875385e-02,\n",
       "         -1.17578461e-03,  -2.37419024e-02,  -2.18893208e-01,\n",
       "          0.00000000e+00,  -5.04674555e-02,  -1.33040678e-01,\n",
       "         -2.24892922e-01,   1.02310964e-01,   1.66752119e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.35371200e-01,\n",
       "          1.08476699e-01,   9.26982786e-02,   2.23322524e-01,\n",
       "          3.66079259e-01,   8.28391376e-01,  -2.07795523e-01,\n",
       "         -3.63161063e-02,   1.54998349e-01,   1.19851313e-01,\n",
       "         -4.06380599e-02,   7.92160868e-02,   0.00000000e+00,\n",
       "          0.00000000e+00,  -3.59510118e-03,   0.00000000e+00,\n",
       "          9.73356913e-02,   3.71562309e-02,  -7.50297422e-02,\n",
       "         -1.03476879e-01,   1.00755570e-01,   0.00000000e+00,\n",
       "         -3.38068449e-01,   0.00000000e+00,   1.38080012e-01,\n",
       "         -5.86846546e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.52946107e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -2.03178659e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.19663096e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.06579904e+00,   1.99521695e-02,\n",
       "          0.00000000e+00,   6.58706806e-03,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -4.40706552e-02,\n",
       "         -6.46765639e-01,   0.00000000e+00,   4.90977994e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -4.16067277e-01,   0.00000000e+00,  -2.34837314e-01,\n",
       "          0.00000000e+00,  -1.00500386e+00,  -1.18941157e-01,\n",
       "          0.00000000e+00,   1.03880157e+00,  -4.95287977e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -6.97939776e-02,  -3.70605890e-02,   1.66213036e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.15845833e-01,\n",
       "          0.00000000e+00,   5.07809988e-02,   0.00000000e+00,\n",
       "         -5.83413731e-01,   6.76377400e-02,   0.00000000e+00,\n",
       "          2.38876965e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.22731199e+00,   3.37696912e-01,\n",
       "          0.00000000e+00,  -5.64509741e-01,   7.25945459e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -4.70624041e-01,  -2.28652647e-01,\n",
       "         -7.01335205e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "         -1.23188018e+00,   0.00000000e+00,  -2.70972677e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.95796697e-01,\n",
       "          0.00000000e+00,   8.72343220e-02,   0.00000000e+00,\n",
       "          0.00000000e+00,   2.77531856e-01,   0.00000000e+00,\n",
       "         -3.22209123e-01,  -1.42122747e-01,   0.00000000e+00,\n",
       "          4.83398892e-01,  -3.31930065e-02,   3.15077093e-01,\n",
       "          0.00000000e+00,   5.96083193e-01,   0.00000000e+00,\n",
       "         -1.14355147e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -7.37929279e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.27905412e-01,   3.66694714e-01,\n",
       "          0.00000000e+00,  -6.57970714e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   1.87137664e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -6.64379165e-02,\n",
       "          0.00000000e+00,   7.91466395e-01,  -4.92342536e-01,\n",
       "          1.51752741e-01,   0.00000000e+00,   1.16172624e-02,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.71701572e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.60086486e-01,   9.95039414e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -5.24908238e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          7.03332231e-01,  -1.19299483e+00,  -2.43279542e-01,\n",
       "          0.00000000e+00,   1.03738907e+00,  -2.58709660e-02,\n",
       "         -2.05019393e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.98220274e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.45702757e+00,\n",
       "          1.64118034e+00,   8.44481378e-01,   3.79388555e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -7.37998358e-01,\n",
       "          0.00000000e+00,   1.60313230e-01,   0.00000000e+00,\n",
       "         -6.32627487e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.07854024e+00,   2.63748800e+00,   0.00000000e+00,\n",
       "          1.62568459e-02,  -2.10438282e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   9.41283895e-01,\n",
       "          2.29884911e+00,   0.00000000e+00,  -5.37713939e-01,\n",
       "          0.00000000e+00,   2.69643025e-01,   0.00000000e+00,\n",
       "          2.49142627e+00,   0.00000000e+00,  -1.80541798e-02,\n",
       "          4.74229597e-01,   0.00000000e+00],\n",
       "       [  8.30836139e-02,   2.68720442e-01,  -1.02030912e-03,\n",
       "          1.01855777e-02,  -5.34818906e-04,   1.56982269e-02,\n",
       "          1.63312832e-03,  -6.74412225e-04,   1.07299766e-01,\n",
       "          9.57528720e-02,   0.00000000e+00,  -3.22227500e-02,\n",
       "          2.20322643e-01,  -7.95571979e-03,  -4.94709891e-02,\n",
       "          0.00000000e+00,   5.74812250e-02,   9.62302589e-02,\n",
       "         -8.24815767e-02,   0.00000000e+00,  -2.35585571e-01,\n",
       "         -2.08083348e-01,  -7.22380007e-01,   2.21643348e-01,\n",
       "         -2.09238228e-02,  -1.11848896e-01,  -4.85063923e-02,\n",
       "          7.98020330e-02,  -1.94770444e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   6.08825915e-02,   0.00000000e+00,\n",
       "         -8.37902944e-02,  -2.19368092e-02,   4.22004429e-02,\n",
       "          1.18452718e-01,  -1.34560493e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -3.59041048e-01,   0.00000000e+00,\n",
       "          4.89440050e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -3.86962230e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -4.07829670e-01,  -1.07286285e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -6.55152274e-02,\n",
       "          4.47653220e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          6.13001052e-01,   0.00000000e+00,   2.93109392e-01,\n",
       "          0.00000000e+00,   1.01480548e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -3.15392178e-01,  -1.33025728e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          5.14604201e-02,   5.43418798e-02,  -4.42212966e-03,\n",
       "          1.12519558e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -6.88354788e-01,   0.00000000e+00,   1.11948158e-01,\n",
       "          0.00000000e+00,  -2.71186061e-02,  -1.12055880e-01,\n",
       "          4.58964046e-01,   1.20945030e-01,   0.00000000e+00,\n",
       "         -2.46316490e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   8.16628288e-01,  -4.68966413e-01,\n",
       "          2.86617141e-01,   0.00000000e+00,  -5.30494707e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   2.94947910e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.47694532e+00,   0.00000000e+00,   2.50337405e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   6.00137594e-01,\n",
       "          0.00000000e+00,  -3.50760332e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,  -4.40009871e-01,   0.00000000e+00,\n",
       "          1.96066428e-01,   6.33808374e-02,   0.00000000e+00,\n",
       "         -5.81743469e-01,  -2.17295491e-02,  -2.63098082e-02,\n",
       "         -2.42325412e-01,  -3.48598933e-03,   0.00000000e+00,\n",
       "          8.52731423e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.10674972e+00,   0.00000000e+00,   1.64033885e-01,\n",
       "          0.00000000e+00,  -4.15128350e-02,  -1.89091188e-01,\n",
       "         -2.76975750e-01,   4.87211859e-01,   3.13562506e-02,\n",
       "          0.00000000e+00,   6.82351972e-02,  -9.34906243e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -8.36523113e-01,   1.20268698e-01,\n",
       "         -2.53437415e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   5.63567259e-01,   0.00000000e+00,\n",
       "         -1.41120182e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.03611566e+00,   6.74272589e-02,\n",
       "         -1.46367199e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          7.71701444e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -5.92549083e-01,   6.81392825e-01,   0.00000000e+00,\n",
       "         -1.09032866e-01,  -1.06444220e+00,   0.00000000e+00,\n",
       "          1.58625743e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   1.21060477e-01,\n",
       "         -6.28227676e-01,  -4.40621322e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   1.25054870e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.05867713e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -2.01231251e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.89597948e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -4.76206703e-01,\n",
       "         -1.81641577e+00,   0.00000000e+00,   5.62731902e-02,\n",
       "         -6.53355321e-02,   0.00000000e+00,   0.00000000e+00,\n",
       "         -1.19914007e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -2.27716493e-01,   0.00000000e+00],\n",
       "       [  6.14180767e-02,   4.35941069e-01,   7.09910005e-05,\n",
       "          3.16130590e-03,  -1.12100182e-03,  -1.17271531e-01,\n",
       "         -7.74619922e-04,   8.31514174e-02,   1.69072799e-01,\n",
       "         -2.23594669e-01,   1.80701610e-01,   4.05362632e-01,\n",
       "          5.48529548e-03,  -2.74739819e-01,  -2.91058617e-01,\n",
       "          1.75496828e-02,  -9.41141355e-02,   8.85960170e-02,\n",
       "         -1.06911289e-01,  -2.34106251e-01,  -6.53649803e-02,\n",
       "         -2.68688819e-01,  -4.65917630e-01,   0.00000000e+00,\n",
       "          2.14548677e-01,  -1.69966850e-01,  -7.60464598e-02,\n",
       "         -5.57625565e-04,   3.13965691e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.00062778e-01,   1.49148317e-02,\n",
       "         -4.46694659e-02,   0.00000000e+00,   1.14233807e-01,\n",
       "          0.00000000e+00,   5.00633339e-02,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   1.61752093e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   3.76388241e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   2.18004409e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   2.84908345e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -7.98065377e-01,   2.22394814e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          1.19773613e-01,   1.66923801e-01,  -1.95570548e-01,\n",
       "         -1.82174887e-01,  -2.41576448e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.95234439e-02,   2.16115418e-01,\n",
       "          0.00000000e+00,  -7.98831834e-01,   0.00000000e+00,\n",
       "          1.29658905e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.10639649e+00,   0.00000000e+00,\n",
       "         -4.22883287e-01,   0.00000000e+00,  -1.95834573e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.55799352e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   8.64800762e-02,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   2.46405252e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          4.46313531e-01,   3.83307198e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.98916366e-01,   0.00000000e+00,\n",
       "          1.45241911e-01,  -8.39766942e-01,   0.00000000e+00,\n",
       "          1.14704484e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -1.38650637e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   5.77471840e-01,  -4.86751105e-01,\n",
       "          3.48643705e-01,   6.64966049e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,  -1.28026380e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   3.38677882e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   5.73378775e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -6.16979319e-01,   0.00000000e+00,\n",
       "         -1.63799182e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -1.53605748e-01,  -1.78460511e-01,   0.00000000e+00,\n",
       "          9.35573973e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -2.12232064e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "         -2.33337036e-01,   7.21251614e-01,   9.28760548e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          2.54048854e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   1.44977338e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   1.98143533e+00,\n",
       "         -1.29062435e+00,  -1.04962528e-01,  -2.61459043e-01,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.01289537e-01,   0.00000000e+00,\n",
       "          9.24600280e-01,   3.19538319e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,  -1.61116356e+00,   0.00000000e+00,\n",
       "         -1.77378990e-02,   1.08027795e-01,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,\n",
       "         -2.84300538e-01,   0.00000000e+00,   3.26446977e-01,\n",
       "          0.00000000e+00,  -2.08704599e-01,   9.09544495e-01,\n",
       "         -3.75448860e-01,   0.00000000e+00,   0.00000000e+00,\n",
       "          0.00000000e+00,   0.00000000e+00]])"
      ]
     },
     "execution_count": 247,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lrcv_L1.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4.2 L2正则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegressionCV(Cs=[0.1, 1, 10, 100, 1000], class_weight=None, cv=5,\n",
       "           dual=False, fit_intercept=True, intercept_scaling=1.0,\n",
       "           max_iter=100, multi_class='ovr', n_jobs=1, penalty='l2',\n",
       "           random_state=None, refit=True, scoring='neg_log_loss',\n",
       "           solver='liblinear', tol=0.0001, verbose=0)"
      ]
     },
     "execution_count": 248,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegressionCV\n",
    "\n",
    "Cs = [0.1, 1, 10,100,1000]\n",
    "\n",
    "# 大量样本（6W+）、高维度（93），L2正则 --> 缺省用lbfgs，为了和GridSeachCV比较，也用liblinear\n",
    "\n",
    "lr_cv_L2 = LogisticRegressionCV(Cs=Cs, cv = 5, scoring='neg_log_loss', penalty='l2', solver='liblinear', multi_class='ovr')\n",
    "lr_cv_L2.fit(X_train, y_train)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: array([[-0.50884062, -0.50872889, -0.51624855, -0.50872682, -0.50871523],\n",
       "        [-0.51465239, -0.51421465, -0.51089936, -0.51566713, -0.51566707],\n",
       "        [-0.50945759, -0.50981512, -0.51188505, -0.50985712, -0.50993141],\n",
       "        [-0.52488348, -0.52490397, -0.52483156, -0.52480481, -0.52488636],\n",
       "        [-0.52286184, -0.52279431, -0.52406801, -0.52406839, -0.52406121]]),\n",
       " 1: array([[-0.48131025, -0.48125904, -0.48123346, -0.48540847, -0.48127521],\n",
       "        [-0.49186198, -0.4918687 , -0.49187182, -0.4918725 , -0.49187199],\n",
       "        [-0.48680081, -0.48744476, -0.48794204, -0.48635295, -0.48820093],\n",
       "        [-0.49345389, -0.49298684, -0.49240574, -0.49250186, -0.4930461 ],\n",
       "        [-0.48317691, -0.4831262 , -0.48313267, -0.48313234, -0.48312424]]),\n",
       " 2: array([[-0.23469144, -0.23696806, -0.23697757, -0.23570412, -0.23647069],\n",
       "        [-0.22099366, -0.218005  , -0.21840655, -0.21843766, -0.2184326 ],\n",
       "        [-0.228083  , -0.2289191 , -0.22834582, -0.22804347, -0.22806868],\n",
       "        [-0.23806273, -0.23867736, -0.23797608, -0.23952612, -0.2399517 ],\n",
       "        [-0.23742365, -0.23846363, -0.23858193, -0.2385946 , -0.23859527]])}"
      ]
     },
     "execution_count": 249,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_cv_L2.scores_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8FfXZ9/HPlYQkBMIa9gTC5oIi\nohEXEMTWulWwLlVb21K11qq11frcj3263K32vtuqhWoBFevSTa211VK1VcQgi4oERBYRSMIW1rCF\nNYQk1/PHOWCIWU5CJpPl+3698uLMzG9mrhly8j2/mTkz5u6IiIg0tLiwCxARkZZJASMiIoFQwIiI\nSCAUMCIiEggFjIiIBEIBIyIigVDAiIhIIBQwIiISCAWMiIgEIiHsAsKUlpbmmZmZYZchItKsLFy4\ncLu7d6utXasOmMzMTHJycsIuQ0SkWTGzdbG00yEyEREJhAJGREQCoYAREZFAKGBERCQQChgREQmE\nAkZERAKhgBERkUAoYESamIMlZfz5/XXs2l8Sdikix0UBI9LE/H5OPj9+ZRkXTZrNf5ZtCbsckXpT\nwIg0IfsPlfL0vDWc2a8zPTokcdufF/Ld5z9kp3oz0gwpYESakOc/WM+uA4f5f5edzCt3jOSei07g\nP8s2c9HEd/j30s1hlydSJwoYkSbiUGkZT87J59wBXTmzX2faxMdx1+cGM/3OUfTqlMx3/rKIO55b\nxI59h8IuVSQmgQaMmV1iZivNLNfM7quh3TVm5maWFR3uambZZrbPzCZXaJdiZq+Z2SdmttzMflVh\n2m1mttTMFpvZXDMbEuS2iTS0lxYWsHXPIe4YO+iY8Sf36sDLt4/k/1x8Im8u38JFk2bz2hL1ZqTp\nCyxgzCwemAJcCgwBbqjqj76ZpQJ3AfMrjC4GfgLcW8WiH3b3k4DhwEgzuzQ6/jl3H+rupwMPAhMb\nbGNEAlZaVs7j7+QxLKMTIwd1/cz0NvFx3DF2EK9+93zSO7fljucW8Z0/L6Rwr3oz0nQF2YMZAeS6\ne767lwAvAOOraPcAkUAoPjLC3fe7+9yK46LjD7h7dvR1CbAISI8O76nQtB3gDbgtIoH615JNbNh5\nkDvHDsLMqm13Ys9U/vGd8/ivS05k5optfGHSO0z/aBPu+nWXpifIgOkDbKgwXBAdd5SZDQcy3P3V\nui7czDoBVwAzK4y7w8zyiATWXfUpWqSxlZc7U7PzOLFHKp87qXut7RPi47j9gkG8dtco+nZtx13P\nf8htf17Itr3Ftc4r0piCDJiqPoYd/ZhlZnHAJOAHdV6wWQLwPPCou+cfXbj7FHcfCPxf4MfVzHur\nmeWYWU5hYWFdVy3S4N78eCurt+3j9rEDiYurvvdS2eAeqfz9tnO579KTyF5ZyBcmzeafizeqNyNN\nRpABUwBkVBhOBzZVGE4FTgVmmdla4Bxg+pET/bWYBqx2999WM/0F4MqqJrj7NHfPcvesbt1qfeKn\nSKDcnSnZufTrmsLlQ3vVef6E+DhuGzOQ1+86n/5p7fjeC4u59U8L2bZHvRkJX5ABswAYbGb9zSwR\nuB6YfmSiuxe5e5q7Z7p7JvA+MM7da3yGsZn9AugIfL/S+MEVBi8HVjfMZogEZ/bq7SzdWMR3xgwk\nIb7+b8dB3dvz0m3n8aPLTmb2qkIumjSbfywqUG9GQhVYwLh7KXAn8AawAnjR3Zeb2f1mNq62+aO9\nmonABDMrMLMhZpYO/IjIVWmLopck3xKd5c7opcuLgXuAbwSwWSINakp2Lr06JnPVGenHvaz4OONb\nowfw+vfOZ1D39tzz4kfc8occtqo3IyGx1vwJJysry3NyauwwiQRmwdqdXPv4e/z0i0O4aVT/Bl12\nWbnzzLw1PPTGSpIS4vjpFadw9Rl9arxCTSRWZrbQ3Ws9naFv8ouEZEp2Ll3bJXLDiL4Nvuz4OOOW\n8wfwn++P5sSeqdz7t4+46dkFbClSb0YajwJGJATLNhYxa2UhN43qT9vE+MDW0z+tHX+99Vz++4oh\nvJe/g4smvcOLORt0bkYahQJGJARTsnNJTU7ga+f2C3xdcXHGN0f25z/fG83JvTrwXy8tYcIzC9i0\n+2Dg65bWTQEj0shyt+3lP8u38I1zM+mQ3KbR1puZ1o4XvnUOPx93Ch+s2cnFk2bz1wXr1ZuRwChg\nRBrZ1Fl5JCfEN/iJ/VjExRnfOC+TN74/mlP6dOD//n0pX3/6AzaqNyMBUMCINKINOw/wz8WbuGFE\nX7q0Swytjr5dU3julnN4YPwpLFy3i4snzea5+erNSMNSwIg0oidm5xFvxq2jB4RdCnFxxtfOjfRm\nTkvvyP97eSlfe+oDCnYdCLs0aSEUMCKNZNueYl7MKeDqM9Pp2TE57HKOyuiSwl9uOZv/+dKpfLg+\n0pv58/vrKC9Xb0aOjwJGpJE8OSef0rJybhsTfu+lMjPjq2f34427RzO8b2d+/MoybnxqPht2qjcj\n9aeAEWkEu/aX8Jf56xk3rDf9urYLu5xqpXdO4U83j+CXVw1lSUERF/92Nn98b616M1IvChiRRvDM\nu2s5UFLGdy4YVHvjkJkZN4zoyxt3j+bMfp356T+Xc8OT77Nux/6wS5NmRgEjErC9xYd5dt4avjCk\nByf2TA27nJj16dSWP940ggevPo2PN+3hkt/O4dl5a9SbkZgpYEQC9pf569lTXModY5t+76UyM+PL\nZ2Xw5j2jOXtAF372r4+5/sn3WbtdvRmpnQJGJEDFh8v4/Zw1nD84jWEZncIup956dWzLMxPO4qFr\nTmPF5j1c8shsnp6r3ozUTAEjEqAXczawfd+hZtl7qczMuDYrgxl3j+G8gWnc/+rHfPmJ98gv3Bd2\nadJEKWBEAnK4rJwn3sknq19nzu7fJexyGkzPjsk89Y0sfnPtMFZt3culj8zh93PyKVNvRipRwIgE\n5OUPN7Jx90HuGDuoxT3oy8y4+sx0ZtwzhvMHp/GL11Zw7ePvkqfejFSggBEJQFm58/isPE7p3YEL\nTuwWdjmB6dEhmSe/nsVvrzudvML9XPbIHKbNzlNvRgAFjEgg/r1sM/nb97fI3ktlZsaVw/sw457R\njDmhG//7+idc8/i75G7bG3ZpEjIFjEgDc3emZOcxoFs7Lj6lZ9jlNJruqck88bUzeeT601mzfT+X\nPTqXx9/Jo7SsPOzSJCQKGJEGlr1yGys27+H2CwYRH9eyey+VmRnjT+/DjLvHcOGJ3fnVvz/h6sff\nY/VW9WZaIwWMSANydya/nUufTm0Zf3rvsMsJTbfUJB678Qwmf2U4G3Ye4PJH5zIlO1e9mVZGASPS\ngN7L38Gi9bu5bcwA2sS37reXmfHF03rz5t2juWhIDx56YyVXPfYuK7eoN9NatO53gEgDm5qdR1r7\nJK7Nygi7lCYjrX0SU756BlO+cgYbdx3ki7+bw+S3V3NYvZkWTwEj0kAWb9jN3NztfOv8/iS3iQ+7\nnCbn8tN68ebdo7n4lJ48/OYqvjR1His27wm7LAmQAkakgUzJzqVj2zZ89Zx+YZfSZHVtn8Tkr5zB\nY189gy1FxYybPJdHZ6o301IpYEQawCdb9jDj4618c2Qm7ZMSwi6nybt0aC/evHsMlw3txcQZqxg/\neR7LNxWFXZY0sEADxswuMbOVZpZrZvfV0O4aM3Mzy4oOdzWzbDPbZ2aTK7RLMbPXzOwTM1tuZr+q\nMO0eM/vYzJaY2Uwz08dIaTRTs/NolxjPhPMywy6l2ejSLpFHrh/OE187k217DzF+8jwmzVhFSal6\nMy1FYAFjZvHAFOBSYAhwg5kNqaJdKnAXML/C6GLgJ8C9VSz6YXc/CRgOjDSzS6PjPwSy3P004CXg\nwYbaFpGarN2+n1eXbOLGc/rRKSUx7HKanYtP6clb94zmimG9eWTmasZNnsuyjerNtARB9mBGALnu\nnu/uJcALwPgq2j1AJAyKj4xw9/3uPrfiuOj4A+6eHX1dAiwC0qPD2e5+INr0/SPjRYL2+Dt5JMTH\ncfP5/cMupdnqlJLIpOtO58mvZ7FjfwlXTpnHxDdXqjfTzAUZMH2ADRWGC6LjjjKz4UCGu79a14Wb\nWSfgCmBmFZNvBv5d12WK1NWm3Qf5+6ICrsvKoHtqctjlNHsXDenBjLtHM+703jz6di7jJs9laYF6\nM81VkAFT1T0yjt5i1czigEnAD+q8YLME4HngUXfPrzTtRiALeKiaeW81sxwzyyksLKzrqkWO8eSc\nfNzh22MGhF1Ki9EpJZGJXz6dpydksetACVdOncdDb3zCodKysEuTOgoyYAqAit82Swc2VRhOBU4F\nZpnZWuAcYPqRE/21mAasdvffVhxpZp8HfgSMc/dDVc3o7tPcPcvds7p1a7m3UZfgbd93iOc/WM+V\nw/uQ3jkl7HJanAtP6sGbd4/hquF9mJKdxxW/m8uSgt1hlyV1EGTALAAGm1l/M0sErgemH5no7kXu\nnubume6eSeS8yTh3z6lpoWb2C6Aj8P1K44cDT0SXsa1hN0Xks56eu4ZDpeV854KBYZfSYnVs24aH\nrh3GM988iz0HS/nS1Hf59X8+ofiwejPNQWAB4+6lwJ3AG8AK4EV3X25m95vZuNrmj/ZqJgITzKzA\nzIaYWTqRHsoQYJGZLTazW6KzPAS0B/4WHT+9ygWLNICig4f503vruOzUXgzs1j7sclq8sSd25817\nRnPNGek8NiuPL/5uLh+u3xV2WVILc2+9T57LysrynJwaO0wiVZr89moefnMVr353FKf26Rh2Oa3K\nO6sKue/vS9i6p5hvjR7A3Z8/QbfmaWRmttDdaz2doW/yi9TRgZJSnp63lrEndlO4hGDMCd148+7R\nXHdWX554J5/LH53DIvVmmiTd00Kkjp7/YAM795dw54WDwi6l1UpNbsMvrxrKZUN7ct/fl3LV1Hfp\nlppEeue2pHdOIb1zWzKi/6Z3bkvvTm3VywmBAkakDg6VljFtdh5n9+/Cmf26hF1Oq3f+4G68cfdo\nnpu/jrxt+ynYfYAlBbv5z7LNHC479vB/99QkMrp8GjrpnVOOhlCvTskkJSiAGpoCRqQO/rFoI1v3\nHOLha4eFXYpEtU9K4NbRx17JV1bubN1TTMGugxTsOnD03w07D7Jo/S5eXbKZsvJPA8gMeqQmR3o+\nlULoSA+otT9Arj4UMCIxKi0r57FZeQxL78ioQWlhlyM1iI8zeneKBMOI/p/taZaWlbN17yE27Dzw\nmRD6YM1O/rn4IBXyhziDnh2SjwZOeue2pHf59FBcz47JCqAqKGBEYvTqks2s33mAH11+JmZV3ahC\nmouE+Dj6dGpLn05tq5x+uKycLUWf9oA2VAih+Wt28koVAdSr47G9nqOH4bq0pWeHZBJaYQApYERi\nUF7uTJ2Vywk92nPRyT3CLkcC1iY+jowuKWR0SQG6fmb64bJyNu8uPvbwW/Tfd/O2s2VPMRW/ARIf\nZ/TqmFzh4oNje0I9OyQTH9fyPrQoYERiMGPFVlZt3ccj159OXAv8QyB10yY+jr5dU+jbtepbBJWU\nlrO56CAbdh57+K1g10Fmry5k655j72SVED2kV7nncySIuqc2zwBSwIjUwt2Zkp1L3y4pXD60V9jl\nSDOQmBBHv67t6Ne1XZXTD5WWsalCD6jiuaBZKwvZtvfYAGoTHwmgjM6fvQAho0sK3donNckPPgoY\nkVrMzd3OkoIifnnV0FZ5HF0aXlJCPP3T2tE/reoAKj5cxsbdB4/p+RwJobdWbGP7vmMDKDE+jj6d\nqzkH1Lkt3VKTQjlvqIARqcXkt3Pp2SGZq87oU3tjkQaQ3Caegd3aV3ufu4MlkQDaUOnwW8HOA7y5\naQs79pcc0z4p4UgAfRo+Y0/szsm9OgS6HQoYkRrkrN3J/DU7+ckXh+iLeNJktE2MZ1D39gzqXnUA\nHSgpZeOuSA+ocggt21jEzv0ldElJVMCIhGlKdi5d2iVyw4iM2huLNBEpiQkM7pHK4B6pVU7fd6iU\nxjhlowPKItVYtrGI7JWF3DQyk5REfRaTlqN9UkKj/E4rYESq8disPFKTEvjauZlhlyLSLClgRKqQ\nu20fry/bzNfP60fHtm3CLkekWVLAiFThsVl5JCXEcdPI/mGXItJsKWBEKtmw8wCvLN7IDSP60rV9\nUtjliDRbChiRSqbNzifO4NbRA8IuRaRZU8CIVLBtTzF/zdnA1Wek06tj1XfaFZHYKGBEKnhq7hpK\ny8q5bczA2huLSI0UMCJRuw+U8Of313HFsN5kVnOPKBGJnQJGJOrZd9eyv6SM2y8YFHYpIi2CAkaE\nyK0znpm3louG9ODEnlXfXkNE6iamgDGzkWbWLvr6RjObaGb9gi1NpPH85f11FB08zB1j1XsRaSix\n9mAeAw6Y2TDgv4B1wB8Dq0qkERUfLuPJOWsYNSiN0zM6hV2OSIsRa8CUursD44FH3P0RQMcRpEX4\nW84Gtu87pN6LSAOLNWD2mtkPgRuB18wsHqj1Bk1mdomZrTSzXDO7r4Z215iZm1lWdLirmWWb2T4z\nm1yhXYqZvWZmn5jZcjP7VYVpo81skZmVmtk1MW6XtHKHy8p5/J18zujbiXMGdAm7HJEWJdaAuQ44\nBNzs7luAPsBDNc0QDaEpwKXAEOAGMxtSRbtU4C5gfoXRxcBPgHurWPTD7n4SMBwYaWaXRsevByYA\nz8W4TSL8c/EmNu4+yJ0XDgrlkbIiLVnMPRgih8bmmNkJwOnA87XMMwLIdfd8dy8BXiByiK2yB4AH\niYQKAO6+393nVhwXHX/A3bOjr0uARUB6dHituy8BymPcJmnlysqdqbNyOblXB8ae2D3sckRanFgD\nZjaQZGZ9gJnAN4Fna5mnD7ChwnBBdNxRZjYcyHD3V2Oso+K8nYArovXUZb5bzSzHzHIKCwvrulpp\nQf6zbAv5hfu5Y+xA9V5EAhBrwJi7HwCuAn7n7l8CTqltnirG+dGJZnHAJOAHMdbw6YLNEoj0oB51\n9/y6zOvu09w9y92zunXrVtdVSwvh7kzJzmVAt3ZcemqvsMsRaZFiDhgzOxf4KvBadFx8LfMUABUf\nZJ4ObKownAqcCswys7XAOcD0Iyf6azENWO3uv42hrchnzFpZyMeb9/CdMQOJb4yHk4u0QrE+lPn7\nwA+Bl919uZkNALJrmWcBMNjM+gMbgeuBrxyZ6O5FQNqRYTObBdzr7jk1LdTMfgF0BG6JsXaRY7g7\nk7Nz6dOpLVcO71P7DCJSLzEFjLu/A7xjZqlm1j56WOquWuYpNbM7gTeI9HaejobT/UCOu0+vaf5o\nr6YDkGhmVwJfAPYAPwI+ARZFj5tPdvffm9lZwMtAZ+AKM/u5u9d2GE9aoflrdrJw3S7uH38KbeJ1\ntySRoMQUMGY2lMg397tEBq0Q+Lq7L69pPnd/HXi90rifVtP2gkrDmdWVU838C4heUSZSkynZuaS1\nT+LLWRm1NxaReov149sTwD3u3s/d+xI5Mf9kcGWJBOOjDbuZs3o7t5zfn+Q2tZ1GFJHjEWvAtDvy\n/RMAd58F6IEZ0uxMyc6lQ3ICN56je7WKBC3WgMk3s5+YWWb058fAmiALE2loK7fs5c2PtzJhZH/a\nJ8V6fYuI1FesAXMT0A34B5ET6d2IfNlSpNl4bFYuKYnxfPO8zLBLEWkVYr2KbBe1XDUm0pSt27Gf\n6R9t4pbzB9C5XWLY5Yi0CjUGjJn9iwrfvq/M3cc1eEUiAXj8nTwS4uO4ZVT/sEsRaTVq68E83ChV\niARoS1ExLy0s4LqzMujeITnsckRajRoDJvoFS5FmbdrsfModvj16YNiliLQqsX7RcimfPVRWBOQA\nv3D3HQ1dmEhD2LHvEM9/sJ7xp/cmo0tK2OWItCqxXqv5b6CMTx/mdT2Rb9QXEblt/xUNXplIA3hm\n3lqKS8u4/QI9DlmkscUaMCPdfWSF4aVmNs/dR5rZjUEUJnK89hQf5g/vreXSU3syqHv7sMsRaXVi\n/R5MezM7+8iAmY0AjrxjSxu8KpEG8Kf31rG3uFS9F5GQxNqDuQV42szaEzk0tge42czaAb8MqjiR\n+jpYUsbTc9dwwYndOLVPx7DLEWmVYv2i5QJgqJl1JPJ0y90VJr8YSGUix+H5D9azY38Jd45V70Uk\nLDEdIjOzjmY2EZgJvGVmv4mGjUiTU1JazrTZ+Yzo34WszC5hlyPSasV6DuZpYC/w5ejPHuCZoIoS\nOR7/WFTAlj3F6r2IhCzWczAD3f3qCsM/N7PFQRQkcjxKy8p57J08hvbpyPmD02qfQUQCE2sP5qCZ\njToyYGYjgYPBlCRSf68t3cy6HQe4Y+wgoo/UFpGQxNqD+Q7whyMn+YGdwISgihKpj/JyZ2p2HoO7\nt+cLQ3qEXY5IqxfrVWSLgWFm1iE6vCfQqkTq4a0VW1m5dS+TrhtGXJx6LyJhq+12/fdUMx4Ad58Y\nQE0idebuTJmVR98uKVxxWu+wyxERau/BpDZKFSLHaV7uDj7asJv//dJQEuJjPbUoIkGq7Xb9P2+s\nQkSOx5TsXHp0SOLqM/uEXYqIRNX5o56ZLQqiEJH6WrhuF+/l7+Bb5w8gKSE+7HJEJKo+xxJ09lSa\nlCnZuXROacNXzu4bdikiUkF9Aua1Bq9CpJ6Wbyri7U+2cdPI/qQkxnrVvYg0hjoHjLv/OIhCROpj\n6qw8UpMS+Pp5mWGXIiKVxHqzy71mtqfSzwYze9nMBtQw3yVmttLMcs3svhraXWNmbmZZ0eGuZpZt\nZvvMbHKFdilm9pqZfWJmy83sVxWmJZnZX6Prmm9mmbFsmzRfeYX7eH3pZr52bj86tm0TdjkiUkms\nxxQmApuIPDLZiDwyuSewksiNMC+oPIOZxQNTgIuAAmCBmU13948rtUsF7gLmVxhdDPwEODX6U9HD\n7p5tZonATDO71N3/DdwM7HL3QWZ2PfBr4LoYt0+aocdn5ZGUEMdNo/qHXYqIVCHWQ2SXuPsT7r7X\n3fe4+zTgMnf/K9C5mnlGALnunu/uJcALwPgq2j0APEgkVABw9/3uPrfiuOj4A+6eHX1dAiwC0qOT\nxwN/iL5+Cfic6WZULVbBrgO8/OFGrj+rL2ntk8IuR0SqEGvAlJvZl80sLvrz5QrTvJp5+gAbKgwX\nRMcdZWbDgQx3fzXmij+dtxNwBZFn1ByzPncvBYqArlXMd6uZ5ZhZTmFhYV1XK03EtNn5mMGto6s9\nQisiIYs1YL4KfA3YBmyNvr7RzNoCd1YzT1W9h6NhZGZxwCTgBzFX++m8CcDzwKPunh/L+o6OcJ/m\n7lnuntWtW7e6rlqagG17i3lhwQauGp5O705twy5HRKoR680u84n0Fqoyt5rxBUBGheF0Iudxjkgl\ncn5lVvRIVk9gupmNc/ecWkqaBqx2999Wsb6CaAB1JHLXZ2lhnpq7htKycm67YGDYpYhIDWK9iuwE\nM5tpZsuiw6eZWW2XKy8ABptZ/+gJ+euB6UcmunuRu6e5e6a7ZwLvA7WGi5n9gkh4fL/SpOnAN6Kv\nrwHedvfqDt9JM1V04DB/fm8dl5/Wm/5p7cIuR0RqEOshsieBHwKHAdx9CZHAqFb0PMidwBvACuBF\nd19uZveb2bjaVmhma4lcvTbBzArMbIiZpQM/AoYAi8xssZndEp3lKaCrmeUC9wDVXhYtzdez765l\nf0kZd4xV70WkqYv1MuUUd/+g0kVZpbXN5O6vA69XGvfTatpeUGk4s5rFVnllmLsXA9fWVpM0X/sP\nlfLMu2v4/Mk9OKlnh7DLEZFaxNqD2W5mA4meNDeza4DNgVUlUoXn5q9n94HD6r2INBOx9mDuIHJi\n/SQz2wisIXJlmUijKD5cxrQ5+Ywc1JXhfav76pWINCWxBsxG4BkgG+gC7CFyQv3+gOoSOcbfFhZQ\nuPcQj1x/etiliEiMYg2YfwK7iXxzflMtbUUa1OGycp54J4/hfTtx7oDPfHdWRJqoWAMm3d0vCbQS\nkWpMX7yJgl0H+fm4U9Ddf0Saj1hP8r9rZkMDrUSkCuXlztRZuZzUM5ULT+oedjkiUgex9mBGEfk+\nyhrgEJFLhd3dTwusMhHgjeVbyCvcz+9uGK7ei0gzE2vAXBpoFSJVcHcmZ+fSP60dlw3tFXY5IlJH\nsd6LbF3QhYhU9s6qQpZv2sODV59GfJx6LyLNTZ0fmSzSWKZk59K7YzJXDu9Te2MRaXIUMNIkzc/f\nwYK1u/j2mIEkJujXVKQ50jtXmqQps/JIa5/IdWdl1N5YRJokBYw0OUsKdjN7VSE3jxpAcpv4sMsR\nkXpSwEiTMyU7lw7JCdx4Tt+wSxGR46CAkSZl9da9vLF8KxPOyyQ1uU3Y5YjIcVDASJMydVYeKYnx\nfHNk/7BLEZHjpICRJmP9jgNM/2gTXz27L53bJYZdjogcJwWMNBmPz84j3oxbzh8Qdiki0gAUMNIk\nbCkq5qWcAq7NSqdHh+SwyxGRBqCAkSbh93PyKXPntjF6HLJIS6GAkdDt3F/CX+avZ/yw3mR0SQm7\nHBFpIAoYCd0z89ZQXFrG7WPVexFpSRQwEqq9xYd59t21XDykJ4O6p4Zdjog0IAWMhOpP769jb3Ep\nd4wdFHYpItLAFDASmoMlZTw1Zw1jTujG0PSOYZcjIg1MASOh+euC9ezYX6Lei0gLpYCRUJSUlvPE\n7HxGZHZhRP8uYZcjIgEINGDM7BIzW2lmuWZ2Xw3trjEzN7Os6HBXM8s2s31mNrlS2/8xsw1mtq/S\n+H5mNtPMlpjZLDNLD2arpCG8/GEBm4uKueNC9V5EWqrAAsbM4oEpwKXAEOAGMxtSRbtU4C5gfoXR\nxcBPgHurWPS/gBFVjH8Y+KO7nwbcD/zyuDZAAlNW7jw2K4+hfToyenBa2OWISECC7MGMAHLdPd/d\nS4AXgPFVtHsAeJBIqADg7vvdfW7FcRWmve/um6tYzhBgZvR1djXrkibgtaWbWbvjAHeMHYiZhV2O\niAQkyIDpA2yoMFwQHXeUmQ0HMtz91QZY30fA1dHXXwJSzaxr5UZmdquZ5ZhZTmFhYQOsVurC3Zma\nncug7u35wpCeYZcjIgEKMmCq+mjqRyeaxQGTgB800PruBcaY2YfAGGAjUPqZAtynuXuWu2d169at\ngVYtsZq5YhufbNnL7RcMJC5rPWfVAAAPPklEQVROvReRliwhwGUXABkVhtOBTRWGU4FTgVnRwyQ9\ngelmNs7dc+q6MnffBFwFYGbtgavdvaietUsA3J3J2bmkd27LuGG9wy5HRAIWZA9mATDYzPqbWSJw\nPTD9yER3L3L3NHfPdPdM4H2gXuECYGZp0V4RwA+Bp4+vfGlo7+XtYPGG3dw2ZiAJ8bpCXqSlC+xd\n7u6lwJ3AG8AK4EV3X25m95vZuNrmN7O1wERggpkVHLkCzcweNLMCICU6/mfRWS4AVprZKqAH8D8N\nvU1yfCZn59I9NYlrztQV5CKtgbl77a1aqKysLM/JqVeHSepo0fpdXDX1XX58+cl6YqVIM2dmC909\nq7Z2Ok4hjWJqdi6dUtpww4i+YZciIo1EASOBcnemf7SJt1Zs46aR/WmXFOR1JSLSlOjdLoFwd97+\nZBsTZ6xi+aY9nNCjPd84LzPsskSkESlgpEG5O7NXb2fijFV8tGE3fbuk8JtrhzH+9N66ckyklVHA\nSIN5NzcSLDnrdtGnU1t+ffVQrjojnTYKFpFWSQEjx23B2p385s2VvJ+/k54dknngylO5LiuDxAQF\ni0hrpoCRevtw/S4mzljFnNXbSWufxH9fMYQbRvQluU182KWJSBOggJE6W1pQxKS3VvH2J9vo0i6R\nH112Mjee04+2iQoWEfmUAkZitmLzHibNWMWbH2+lY9s2/J+LT2TCeZm69FhEqqS/DFKr1Vv38tu3\nVvPa0s2kJiVw9+dP4KZRmaQmtwm7NBFpwhQwUq38wn08OnM1//xoEylt4vnuhYO4ZdQAOqYoWESk\ndgoY+Yz1Ow7w6NurefnDjSTGx3Hr6AF8e/RAurRLDLs0EWlGFDBy1MbdB5n89mr+llNAfJwx4bxM\nbhszkG6pSWGXJiLNkAJG2FJUzNRZubzwQeQJ1185uy93jB1Ejw7JIVcmIs2ZAqYVK9x7iMdm5fHn\n+esoL3euzcrgzgsH0adT27BLE5EWQAHTCu3cX8ITs/P447vrKCkr56rhfbjrc4PJ6JISdmki0oIo\nYFqRogOHeXJOPs/MW8OBw2VceXokWPqntQu7NBFpgRQwrcCe4sM8PXcNT81Zw95DpVx+Wi/u/vxg\nBnVPDbs0EWnBFDAt2P5DpTz77lqmzc6n6OBhLj6lB9///Amc3KtD2KWJSCuggGmBDpaU8af31/L4\nO/ns3F/ChSd1556LTuDUPh3DLk1EWhEFTAtSfLiM5+avZ+qsPLbvO8T5g9O456ITGN63c9iliUgr\npIBpAQ6VlvFiTgFT3s5ly55izh3QlcduPIOzMruEXZqItGIKmGbscFk5f19YwO/ezmXj7oNk9evM\nxOuGcd7AtLBLExFRwDRHpWXlvLJ4E4/OXM36nQcYltGJX141lPMHp2FmYZcnIgIoYJqVsnLn1SWb\neOSt1eRv388pvTvw1DeyuPCk7goWEWlyFDDNQHm585/lW/jtW6tYtXUfJ/VM5fEbz+TiU3ooWESk\nyVLANGHuzoyPtzLprdWs2LyHgd3aMfkrw7ns1F7ExSlYRKRpiwty4WZ2iZmtNLNcM7uvhnbXmJmb\nWVZ0uKuZZZvZPjObXKnt/5jZBjPbV2l83+g8H5rZEjO7LJitCp67k71yG+OnzOPWPy3kYEkpk64b\nxpt3j+GLp/VWuIhIsxBYD8bM4oEpwEVAAbDAzKa7+8eV2qUCdwHzK4wuBn4CnBr9qehfwGRgdaXx\nPwZedPfHzGwI8DqQ2TBb0zjcnXm5O5g4YyWL1u8mvXNbHrzmNK4a3oeE+EA/C4iINLggD5GNAHLd\nPR/AzF4AxgMfV2r3APAgcO+REe6+H5hrZoMqL9Td348u7zOTgCP3QOkIbDr+TWg88/N38JsZq/hg\nzU56d0zmf780lGvOTCcxQcEiIs1TkAHTB9hQYbgAOLtiAzMbDmS4+6tmdi/H52fAm2b2XaAd8Pmq\nGpnZrcCtAH379j3OVR6/het2MXHGSubl7qB7ahI/H3cK14/IICkhPuzSRESOS5ABU9WJAj860SwO\nmARMaKD13QA86+6/MbNzgT+Z2anuXn5MAe7TgGkAWVlZXsVyGsVHG3Yz6a1VzFpZSFr7RH58+cnc\neE4/ktsoWESkZQgyYAqAjArD6Rx72CqVyPmVWdHDXT2B6WY2zt1z6rG+m4FLANz9PTNLBtKAbfVY\nVmCWbypi0ozVvLViK51T2nDfpSfx9XP7kZKoC/pEpGUJ8q/aAmCwmfUHNgLXA185MtHdi4gEAABm\nNgu4t57hArAe+BzwrJmdDCQDhfVcVoNbtXUvk2as4t/LttAhOYEfXHQCE0ZmkprcJuzSREQCEVjA\nuHupmd0JvAHEA0+7+3Izux/IcffpNc1vZmuJnLRPNLMrgS+4+8dm9iCRoEoxswLg9+7+M+AHwJNm\ndjeRQ3ET3D20Q2BH5BXu45G3VvOvJZtol5jAXZ8bzM2j+tOxrYJFRFo2awJ/g0OTlZXlOTn17TDV\nbN2O/TwyczWvfLiR5DbxTDgvk2+dP4DO7RIDWZ+ISGMxs4XunlVbOx34b2AFuw7wu5m5vLSogIQ4\n4+ZR/fn2mIGktU8KuzQRkUalgGkgm4sOMiU7l78u2IBhfO2cftx+wUC6d0gOuzQRkVAoYI7Ttr3F\nTM3O47kP1uPufDkrgzsvHESvjm3DLk1EJFQKmHrase8QT8zO54/vreVwmXPNGenceeEgMrqkhF2a\niEiToICphxcXbOBn/1pO8eEyrhzeh7suHExmWruwyxIRaVIUMPWQ3qUtnzu5B9/73GAGdW8fdjki\nIk2SAqYezhuYpufei4jUQrfqFRGRQChgREQkEAoYEREJhAJGREQCoYAREZFAKGBERCQQChgREQmE\nAkZERALRqp8HY2aFwLp6zp4GbG/AchqK6qob1VV3TbU21VU3x1NXP3fvVlujVh0wx8PMcmJ54E5j\nU111o7rqrqnWprrqpjHq0iEyEREJhAJGREQCoYCpv2lhF1AN1VU3qqvummptqqtuAq9L52BERCQQ\n6sGIiEggFDAxMrNrzWy5mZWbWbVXXpjZJWa20sxyzey+Rqiri5nNMLPV0X87V9OuzMwWR3+mB1hP\njdtvZklm9tfo9PlmlhlULXWsa4KZFVbYR7c0Ul1Pm9k2M1tWzXQzs0ejdS8xszOaSF0XmFlRhf31\n00aoKcPMss1sRfS9+L0q2jT6/oqxrkbfX9H1JpvZB2b2UbS2n1fRJrj3pLvrJ4Yf4GTgRGAWkFVN\nm3ggDxgAJAIfAUMCrutB4L7o6/uAX1fTbl8j7KNatx+4HXg8+vp64K9NpK4JwOQQfq9GA2cAy6qZ\nfhnwb8CAc4D5TaSuC4BXG3lf9QLOiL5OBVZV8f/Y6PsrxroafX9F12tA++jrNsB84JxKbQJ7T6oH\nEyN3X+HuK2tpNgLIdfd8dy8BXgDGB1zaeOAP0dd/AK4MeH01iWX7K9b7EvA5M7MmUFco3H02sLOG\nJuOBP3rE+0AnM+vVBOpqdO6+2d0XRV/vBVYAfSo1a/T9FWNdoYjuh33RwTbRn8on3gN7TypgGlYf\nYEOF4QKC/0Xr4e6bIfKLDnSvpl2ymeWY2ftmFlQIxbL9R9u4eylQBHQNqJ661AVwdfSwyktmlhFw\nTbEK43cqVudGD73828xOacwVRw/jDCfyibyiUPdXDXVBSPvLzOLNbDGwDZjh7tXus4Z+TyY0xEJa\nCjN7C+hZxaQfufs/Y1lEFeOO+zK9muqqw2L6uvsmMxsAvG1mS90973hrqySW7Q9kH9UilnX+C3je\n3Q+Z2W1EPtFdGHBdsQhjf8ViEZHbhewzs8uAV4DBjbFiM2sP/B34vrvvqTy5ilkaZX/VUldo+8vd\ny4DTzawT8LKZneruFc+tBbbPFDAVuPvnj3MRBUDFT77pwKbjXGaNdZnZVjPr5e6bo4cCtlWzjE3R\nf/PNbBaRT1kNHTCxbP+RNgVmlgB0JPhDMbXW5e47Kgw+Cfw64JpiFcjv1PGq+AfU3V83s6lmlubu\ngd5zy8zaEPkj/hd3/0cVTULZX7XVFdb+qlTD7uh7/xKgYsAE9p7UIbKGtQAYbGb9zSyRyAmzwK7Y\nipoOfCP6+hvAZ3paZtbZzJKir9OAkcDHAdQSy/ZXrPca4G2Pnl0MUK11VTpOP47IcfSmYDrw9ejV\nUecARUcOiYbJzHoeOU5vZiOI/C3ZUfNcx71OA54CVrj7xGqaNfr+iqWuMPZXdF3doj0XzKwt8Hng\nk0rNgntPNvZVDc31B/gSkaQ/BGwF3oiO7w28XqHdZUSuIskjcmgt6Lq6AjOB1dF/u0THZwG/j74+\nD1hK5OqppcDNAdbzme0H7gfGRV8nA38DcoEPgAGN9P9XW12/BJZH91E2cFIj1fU8sBk4HP39uhm4\nDbgtOt2AKdG6l1LNFYwh1HVnhf31PnBeI9Q0isihmyXA4ujPZWHvrxjravT9FV3vacCH0dqWAT+N\njm+U96S+yS8iIoHQITIREQmEAkZERAKhgBERkUAoYEREJBAKGBERCYQCRiRgZrav9lY1zv9S9A4M\nmFl7M3vCzPKid8edbWZnm1li9LW+PC1NhgJGpAmL3rMq3t3zo6N+T+Rb1oPd/RQid4FO88hNPGcC\n14VSqEgVFDAijST67fKHzGyZmS01s+ui4+Oitw5ZbmavmtnrZnZNdLavEr07g5kNBM4Gfuzu5RC5\n9Y+7vxZt+0q0vUiToO60SOO5CjgdGAakAQvMbDaRW/dkAkOJ3A17BfB0dJ6RRL5VD3AKsNgjNy+s\nyjLgrEAqF6kH9WBEGs8oIndsLnP3rcA7RAJhFPA3dy939y1EblVzRC+gMJaFR4OnxMxSG7hukXpR\nwIg0nuoe4lTTw50OErlXFETuZTXMzGp63yYBxfWoTaTBKWBEGs9s4LroA6C6EXks8QfAXCIPO4sz\nsx5EHq97xApgEIBHnt+TA/y8wp15B5vZ+OjrrkChux9urA0SqYkCRqTxvEzkrrYfAW8D/xU9JPZ3\nIncsXgY8QeRpiEXReV7j2MC5hcjD53LNbCmRZ9cced7JWOD1YDdBJHa6m7JIE2Bm7T3ytMOuRHo1\nI919S/QZHtnR4epO7h9Zxj+AH7r7ykYoWaRWuopMpGl4NfpgqETggWjPBnc/aGb/TeS56eurmzn6\nILVXFC7SlKgHIyIigdA5GBERCYQCRkREAqGAERGRQChgREQkEAoYEREJhAJGREQC8f8BdMo8eSqZ\nKn8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f399624e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C is: [  1.   10.    0.1]\n"
     ]
    }
   ],
   "source": [
    "n_Cs = len(Cs)\n",
    "n_classes = 3\n",
    "scores =  np.zeros((n_classes,n_Cs))\n",
    "\n",
    "for j in range(n_classes):\n",
    "        scores[j][:] = np.mean(lr_cv_L2.scores_[j],axis = 0)\n",
    "    \n",
    "mse_mean = -np.mean(scores, axis = 0)\n",
    "plt.plot(np.log10(Cs), mse_mean.reshape(n_Cs,1)) \n",
    "#plt.plot(np.log10(reg.Cs)*np.ones(3), [0.28, 0.29, 0.30])\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('neg-logloss')\n",
    "plt.show()\n",
    "\n",
    "print ('C is:',lr_cv_L2.C_)  #对多类分类问题，每个类别的分类器有一个C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.5  SVM\n",
    "### 4.5.1 线性SVM正则超参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性SVM LinearSVC的需要调整正则超参数包括C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1）  \n",
    "采用交叉验证，网格搜索步骤与Logistic回归正则参数处理类似.  \n",
    "这里我们用校验集（X_val、y_val）来估计模型性能  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# 训练样本太大，交叉验证太慢，用train_test_split估计模型性能\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train_part, X_val, y_train_part, y_val = train_test_split(X_train, y_train, train_size = 0.8,random_state = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#LinearSVC不能得到每类的概率，在Otto数据集要求输出每类的概率，这里只是示意SVM的使用方法\n",
    "#https://xacecask2.gitbooks.io/scikit-learn-user-guide-chinese-version/content/sec1.4.html\n",
    "#1.4.1.2. 得分与概率\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.metrics import accuracy_score\n",
    "import sklearn.metrics\n",
    "\n",
    "SVC1 = LinearSVC().fit(X_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.90      0.01      0.01      1396\n",
      "          1       0.23      1.00      0.37       454\n",
      "          2       0.00      0.00      0.00       150\n",
      "\n",
      "avg / total       0.68      0.23      0.09      2000\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[   9 1387    0]\n",
      " [   0  454    0]\n",
      " [   1  149    0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "y_predict = SVC1.predict(X_val)\n",
    "\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\"% (SVC1, classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % confusion_matrix(y_val, y_predict))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fit_grid_point_Linear(C, X_train, y_train, X_val, y_val):\n",
    "    # 在训练集是那个利用SVC训练\n",
    "    SVC2 =  LinearSVC( C = C)\n",
    "    SVC2 = SVC2.fit(X_train, y_train)\n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC2.score(X_val, y_val)\n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.701\n",
      "accuracy: 0.6185\n",
      "accuracy: 0.6975\n",
      "accuracy: 0.697\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuYVPWd5/H3F5oGFY0ISJCLgLQO\noIPEFlQUqINmcHei487GaLJPYuaJTnZixmQ2F91MNhnzZHd2s5vsZsfMM+Yy2WQmcUxM1Ekyg6eg\nuQpKo6ICoggq4AVEQFDufPeP3ylpmm6q6K5Tpy6f1/PUU13Vp6q+R6E+nN/V3B0REZET6ZN1ASIi\nUv0UFiIiUpTCQkREilJYiIhIUQoLEREpSmEhIiJFKSxERKQohYWIiBSlsBARkaKasi6gXIYMGeJj\nxozJugwRkZqycuXKN919aLHj6iYsxowZQ3t7e9ZliIjUFDN7uZTj1AwlIiJFKSxERKQohYWIiBSl\nsBARkaIUFiIiUpTCQkREilJYiIhIUamGhZnNMbN1ZrbezO7s4vffMbOnktvzZrazw+8+YWYvJLdP\npFmnSLnt2QOrVsEDD8C3vgVr12ZdkUjvpDYpz8z6AvcA1wCbgRVm9rC7rykc4+6f73D8Z4Epyc9n\nAV8DWgEHViav3ZFWvSIna/duWL8+3F544difX3/92GOXLIGHHsqmTpFySHMG91RgvbtvADCz+4Dr\ngTXdHH8zISAA/gCI3f2t5LUxMAf4eYr1ihxn167uA2Hr1mOPff/7oaUFrr0Wxo8Pt5YW+M534OGH\n4fBh6Ns3m/MQ6a00w2IEsKnD483AtK4ONLNzgbHA/BO8dkQKNYqwY0f3gfDmm8ceO2JECIEPfeho\nGIwfD+edBwMHdv3+114LP/0pPPkktLamfz4iaUgzLKyL57ybY28Cfunuh0/mtWZ2G3AbwOjRo3tS\nozSI7du7D4S33jr22FGjQgDccMOxgTBuHJx22sl/9qxZ4X7+fIWF1K40w2IzMKrD45HAq90cexPw\nmU6vndXptQs6v8jd7wXuBWhtbe0uiKQBuIergK4CYf36cPVQYAajR4cA+PCHjw+EU04pb23Dh8OE\nCdDWBl/6UnnfW6RS0gyLFUCLmY0FthAC4aOdDzKzC4BBwLIOT88F/quZDUoefxC4K8VapQa4h36C\n7gJh166jx/bpA+eeGwLgppuOhsH48TB2LAwYUNnaowh+/GM4cACamyv72SLlkFpYuPshM7ud8MXf\nF/iRu682s7uBdnd/ODn0ZuA+d/cOr33LzL5BCByAuwud3eW2fz/87ndpvLP01s6dxwfC7t1Hf9+3\nL4wZEwLgssuOD4Rq+lLO5eCee2DFCpg+PetqRE6edfiOrmmtra3ek/0stm2Ds89OoSApi6am8MXf\ncXRR4ecxY6Bfv6wrLM327TBkCNx9N3z1q1lXI+7w2mvhCrS5+eitX7/QTNlIzGyluxftTaubzY96\natAgeOqprKuQrgwcGJqSmurgT+ngwXDxxaHfQmGRve9/H/70T7v+Xb9+xwZIT279+/f+PYrdKj0M\nuw7+GvZOUxNMnpx1FdIIcjn43vdg797yd6LLyXnoofAPkTvvDP1IXd327+/+d4XbO+8UP+bAgXTO\noW/fo8ExdSo88kg6n1PQ8GEhUilRFCboLVsWfpZsHDgACxfCLbfApz+d/ue5w6FDpYXKyQRVx9vI\nkemfh8JCpEJmzAj/GmxrU1hkafnycEVwzTWV+Tyz0LTVr1/P5ulUC606K1IhZ5wBl1wSJudJduI4\nhHZhsqSURmEhUkFRBI8/HlallWzk86GN/33vy7qS2qKwEKmgKArt10uWZF1JY9q5M4T11VdnXUnt\nUViIVND06aHtWk1R2ViwAI4cqVx/RT1RWIhU0KmnhtnmCotsxHGYv3PZZVlXUnsUFiIVFkVhufId\n2sqr4uIYZs6snZn/1URhIVJhURSaQhYtyrqSxvLyy2GtMTVB9YzCQqTCpk0Lq96qKaqy8vlwr87t\nnlFYiFRY//5w5ZUKi0rL5+Gcc2DixKwrqU0KC5EMRBE8++zx+3hLOo4cCWFx9dWNt6psuSgsRDKQ\ny4X7BQsyLaNhrFoVdlJUE1TPKSxEMtDaCqefrqaoSlF/Re8pLEQy0NQUFhZsa8u6ksYQx3DhhWE/\ndOkZhYVIRqIInn8eNm/OupL6tm8fLF6sq4reUliIZKTQb6Gri3QtXRoCQ/MrekdhIZKRyZPDtr4K\ni3TFcZixPWNG1pXUNoWFSEb69AlXF/Pmhd3UJB35PFx+eVgTSnpOYSGSoVwOXnkFNm7MupL6tH07\nPPGEmqDKQWEhkqHC9qpqikrH/Pnhqk2d272nsBDJ0IQJMGyY5lukJY7DjnitrVlXUvsUFiIZMgtN\nUYV/AUv5uIewyOXCvBbpHYWFSMaiCF5/HZ57LutK6suGDfDSS+qvKBeFhUjG1G+RjjgO9wqL8lBY\niGRs3DgYPVr9FuUWx+G/6/jxWVdSHxQWIhkr9Fu0tYWltKX3Dh8O4XvNNVqSvFwUFiJVIIrgrbfg\nmWeyrqQ+rFwJO3dqyGw5KSxEqkBhnSg1RZVHYUny2bOzraOeKCxEqsCoUaFtXWFRHnEMU6bA0KFZ\nV1I/FBYiVSKKYNEiOHQo60pq2zvvhJVm1QRVXgoLkSoRRfD222EtI+m5xYvh4EENmS03hYVIlZg1\nK9yrKap34hj694crr8y6kvqisBCpEsOGwaRJCoveyudDUJxyStaV1BeFhUgViSJYsgQOHMi6ktr0\nxhvw9NNqgkqDwkKkikQR7N0Ljz2WdSW1qTBkVp3b5aewEKkiM2eGGcdqiuqZfB7OOisMm5XySjUs\nzGyOma0zs/Vmdmc3x9xoZmvMbLWZ/azD8/8jeW6tmX3XTJP2pf4NGhS+6LSo4MkrLEk+e3bYslbK\nK7X/pGbWF7gHuBaYCNxsZhM7HdMC3AVMd/dJwOeS568ApgO/D1wIXArMTKtWkWoSRbBsGbz7btaV\n1JZ162DLFvVXpCXN/J0KrHf3De5+ALgPuL7TMbcC97j7DgB335o878AAoBnoD/QD3kixVpGqkcuF\nDu5HH826ktqiJcnTlWZYjAA2dXi8OXmuo/OB881sqZktN7M5AO6+DGgDXktuc919bYq1ilSNq66C\nvn3VFHWy4hjOOw/GjMm6kvqUZlh01cfQeePIJqAFmAXcDPzAzM40s/HABGAkIWAiM5tx3AeY3WZm\n7WbWvm3btrIWL5KV00+HqVPVyX0yDh6EBQt0VZGmNMNiMzCqw+ORwKtdHPOQux90943AOkJ43AAs\nd/c97r4H+Bfgss4f4O73unuru7cO1YphUkdyOVixAnbvzrqS2vD44+G/lYbMpifNsFgBtJjZWDNr\nBm4CHu50zINADsDMhhCapTYArwAzzazJzPoROrfVDCUNI4rCBj6LF2ddSW3I58MIqMIWtVJ+qYWF\nux8CbgfmEr7o73f31WZ2t5ldlxw2F9huZmsIfRRfdPftwC+BF4FngFXAKnf/57RqFak2V1wBzc1q\niipVHENraxh6LOkw987dCLWptbXV29vbsy5DpGxyOdi1S6vQFvP222Ei3pe/DN/8ZtbV1B4zW+nu\nrcWO09QVkSqVy8FTT4XtVqV7CxeGJjt1bqdLYSFSpaIozEpeuDDrSqpbHMOpp8Lll2ddSX1TWIhU\nqalTw5eg+i1OLJ+HGTPCHhaSHoWFSJVqbg77Migsurd5M6xdqyaoSlBYiFSxKII1a8I+DXI8LUle\nOQoLkSpWmDegpT+6ls+HHQYvuijrSuqfwkKkik2ZAmecoaaorriHsLj66rAHiKRLYSFSxZqawoZI\nurI43jPPhOY5NUFVhsJCpMpFEaxfD6+8knUl1UX9FZWlsBCpcuq36Focw+/9HowcmXUljUFhIVLl\nLrwQBg9WWHS0fz8sWqQhs5WksBCpcn36hKU/5s8PnbpydNtZhUXlKCxEakAUwaZN8OKLWVdSHeI4\n7CY4c2bWlTQOhYVIDcjlwr2G0Ab5PFx2WRhWLJWhsBCpARdcAMOHq98CYMcOaG/XKKhKU1iI1ACz\n0BSlfosQmEeOqL+i0hQWIjUil4OtW8NaUY0sjuH008OqvFI5CguRGqH5FkEcw6xZ0K9f1pU0FoWF\nSI0YOxbGjGnsTu6NG8OIMDVBVZ7CQqSGRBEsWBDa7BuRlvjIjsJCpIbkcmE00KpVWVeSjXweRowI\ny3xIZSksRGpII8+3OHIE5s0LTVBakrzyFBYiNWTEiDDnohHD4sknYft2NUFlpaSwMLMHzOzfmpnC\nRSRjuVxYRO/gwawrqSz1V2Sr1C//vwU+CrxgZn9tZmoxFMlIFMGePbByZdaVVFYch+1Thw3LupLG\nVFJYuHve3T8GfAB4CYjN7FEz+6SZabSzSAXNmhXuG6kpau9eWLJEQ2azVHKzkpkNBm4BPgU8Cfwf\nQnjEqVQmIl0aOjT8C7uRwmLJkrCHhcIiO6X2WfwKWAycCnzI3a9z939y988CA9MsUESOF0WwdGn4\nAm0EcQzNzXDVVVlX0rhKvbL4G3ef6O7/zd1f6/gLd29NoS4ROYEogn37YPnyrCupjHwerrgCTjst\n60oaV6lhMcHMziw8MLNBZvZnKdUkIkXMmBF20GuEpqht28KwWTVBZavUsLjV3XcWHrj7DuDWdEoS\nkWLOPBM+8IHGWFRw3rxwryGz2So1LPqYHZ0zaWZ9geZ0ShKRUkRRaIZ6552sK0lXPh/C8ZJLsq6k\nsZUaFnOB+81stplFwM+Bf02vLBEpJorCxLylS7OuJD3uoXM7isKe25KdUsPiy8B84D8CnwHmAV9K\nqygRKW76dGhqqu+mqPXr4ZVX1F9RDZpKOcjdjxBmcf9tuuWISKkGDoRp0+q7kztOZnEpLLJX6jyL\nFjP7pZmtMbMNhVvaxYnIiUURtLfDrl1ZV5KOOA4bPo0bl3UlUmoz1N8TrioOATngJ8BP0ypKREqT\ny4WluxctyrqS8jt0KDSxaUny6lBqWJzi7vMAc/eX3f3rQJReWSJSissvh/7967PfonDFpCGz1aGk\nPgtgX7I8+QtmdjuwBTg7vbJEpBQDBoSO7nrst8jnwxXF7NlZVyJQ+pXF5wjrQv05cAnwH4BPFHuR\nmc0xs3Vmtt7M7uzmmBuTvpDVZvazDs+PNrNHzGxt8vsxJdYq0lCiKGyz+uabWVdSXnEcJh4OHpx1\nJQIlhEUyAe9Gd9/j7pvd/ZPu/sfufsJVaZLX3QNcC0wEbjaziZ2OaQHuAqa7+yRCKBX8BPiWu08A\npgJbT+bERBpFYavVhQuzraOc9uyBZcvUBFVNioaFux8GLuk4g7tEU4H17r7B3Q8A9wHXdzrmVuCe\nZPkQ3H0rQBIqTe4eJ8/vcfd3T/LzRRrCpZeGBfbqqSmqsBOghsxWj1L7LJ4EHjKzXwDvLS7g7r86\nwWtGAJs6PN4MTOt0zPkAZrYU6At83d3/NXl+Z7I0+lggD9yZBNd7zOw24DaA0aNHl3gqIvWlX7+w\nsGA9hUUcH+2PkepQap/FWcB2wgioDyW3Pyzymq6uRLzT4yagBZgF3Az8IFndtgm4CvgCcCkwjrDx\n0rFv5n6vu7e6e+vQoUNLPBWR+pPLwXPPwWuvFT+2FuTzYe+KAQOyrkQKSp3B/ckevPdmYFSHxyOB\nV7s4Zrm7HwQ2mtk6QnhsBp509w0AZvYgcBnwwx7UIVL3omQge1sbfPSj2dbSW6+9Bs8+Cx//eNaV\nSEclhYWZ/T3HXxXg7n9ygpetAFrMbCxhqO1NQOc/xg8Srih+bGZDCM1PG4CdwCAzG+ru2whXNO2l\n1CrSiC6+OKzMOn9+7YdFPh/u1bldXUrts/hNh58HADdw/FXCMdz9UDInYy6hP+JH7r7azO4G2t39\n4eR3HzSzNcBh4Ivuvh3AzL4AzEs61lcC3z+J8xJpKH37wsyZ9TE5L5+HIUNg8uSsK5GOzP24C4bi\nLwoT9PLuXjWzuFtbW729XRcf0ri++1244w7YuDGsp1SL3GHEiNBhf999WVfTGMxsZSnbY5fawd1Z\nC6DhRyJVpGO/Ra1auzb0WWjIbPUpddXZ3Wb2duEG/DNhjwsRqRKTJsHQobU9hFZLklevUkdDnZ52\nISLSO2ZhCG1bW2jOqcWVWuMYWlpA06aqT6lXFjeY2fs6PD7TzP4ovbJEpCeiCLZsgRdeyLqSk3fw\nICxYoKuKalVqn8XX3P297VXcfSfwtXRKEpGeKvRb1GJT1PLl8M47GjJbrUoNi66OK3XYrYhUyPjx\nYTRRLXZy5/PQp8/RhRGlupQaFu1m9m0zO8/MxpnZdwhzH0SkipiFq4u2trCDXi2JY5g6NUwulOpT\nalh8FjgA/BNwP7AX+ExaRYlIz0URbNsGq1dnXUnpdu2Cxx9XE1Q1K3U01DtAl5sXiUh1KTTjtLXB\nRRdlW0upFiyAw4fVuV3NSh0NFSerwRYeDzKzuemVJSI9de65MG5cbXVyx3HYk+Oyy7KuRLpTajPU\nkGQEFADJZkXag1ukSkXR0X+t14I4DmtbNTdnXYl0p9SwOGJm702TSfbDPvlFpUSkIqIo9AM8+WTW\nlRT3yivw/PNqgqp2pQ5//QqwxMwKu/zOINmhTkSqz6xZ4b6tDVqLLhGXLS1JXhtKurJItjptBdYR\nRkT9J8KIKBGpQsOHw4QJtdFvkc+HeidNyroSOZFSNz/6FHAHYbe7pwi71i0jbEokIlUoiuDHPw7L\naPTrl3U1XTtyJITFnDm1uZZVIym1z+IOwl7YL7t7DpgCbEutKhHptVwuLJ+xYkXWlXTv6afDnBA1\nQVW/UsNin7vvAzCz/u7+HHBBemWJSG8V+i2quSlK/RW1o9Sw2JzMs3gQiM3sIYpsqyoi2Ro8OOzN\nXc1hEccwcSKcc07WlUgxpXZw3+DuO93968BXgR8CWqJcpMrlcvDoo7BvX9aVHG/fPli8WENma8VJ\nb6vq7gvd/WF3P5BGQSJSPlEE+/fDsmVZV3K8Rx+FvXsVFrWip3twi0gNmDED+vatzqaoOIamplCj\nVD+FhUgdO+MMuOSS6tzfIp+Hyy+H07Vpc01QWIjUuSiCxx6DPXuyruSo7dth5UqNgqolCguROhdF\ncOgQLFmSdSVHtbWBu/oraonCQqTOTZ8eZnBXU79FHIcmsksvzboSKZXCQqTOnXpq2Ceimvot4jgM\n620qdSlTyZzCQqQBRBE88QTs2JF1JbBhA2zcqCaoWqOwEGkAURQW7Vu0KOtKwlUFqHO71igsRBrA\ntGkwYEB1NEXl8zBqFJx/ftaVyMlQWIg0gP794cors+/kPnwY5s0LTVBakry2KCxEGkQUwTPPhCXB\ns1LoN1ETVO1RWIg0iFwu3C9YkF0NhSXJZ8/OrgbpGYWFSINobQ1La2TZFBXHMHkynH12djVIzygs\nRBpEYdG+rMLi3Xdh6VINma1VCguRBhJF8PzzsGVL5T978WI4cEBhUasUFiINpNBvkcUQ2jiG5uYw\nKktqj8JCpIFMngyDBmXTFJXPh6A49dTKf7b0nsJCpIH06ROuLiodFm+8AatWachsLVNYiDSYXA5e\nfjmsz1Qp8+aFe/VX1K5Uw8LM5pjZOjNbb2Z3dnPMjWa2xsxWm9nPOv3uDDPbYmZ/k2adIo0kisJ9\nJa8u8nk46yyYMqVynynllVpYmFlf4B7gWmAicLOZTex0TAtwFzDd3ScBn+v0Nt8AFqZVo0gjmjAB\nhg2rXFi4h87tKAr7gUttSvPKYiqw3t03uPsB4D7g+k7H3Arc4+47ANx9a+EXZnYJMAx4JMUaRRqO\nWWiKKuxWl7bnn4fNm9UEVevSDIsRwKYOjzcnz3V0PnC+mS01s+VmNgfAzPoA/wv44ok+wMxuM7N2\nM2vfluWCNyI1Jorgtddg3br0P0tLkteHNMOiqzUlO/87pgloAWYBNwM/MLMzgT8DfufumzgBd7/X\n3VvdvXXo0KFlKFmkMVSy3yKOYdy4cJPalWZYbAZGdXg8Eni1i2MecveD7r4RWEcIj8uB283sJeB/\nAh83s79OsVaRhjJuHIwenf7kvEOHwmeoCar2pRkWK4AWMxtrZs3ATcDDnY55EMgBmNkQQrPUBnf/\nmLuPdvcxwBeAn7h7l6OpROTkdey3OHIkvc95/HHYvVtNUPUgtbBw90PA7cBcYC1wv7uvNrO7zey6\n5LC5wHYzWwO0AV909+1p1SQiR0URbN8e9rhISz4fgqnQ7CW1y7wSwyEqoLW11dvb27MuQ6RmbNoU\nmqK+/W34/OfT+YyrroL9+8MVhlQnM1vp7q3FjtMMbpEGNWoUjB+fXr/F7t2wfLmaoOqFwkKkgUUR\nLFwYOqLLrfC+6tyuDwoLkQYWRfD222Fv7HKLYzjlFLjiivK/t1SewkKkgc2aFe7TaIrK58POfP37\nl/+9pfIUFiINbNgwmDSp/JPztmyBNWvUBFVPFBYiDS6KYMmSsOVpueTz4V6d2/VDYSHS4KII3n23\nvMNb83k4+2y46KLyvadkS2Eh0uBmzgwT58rVFOUewmL27LAzn9QH/a8UaXCDBoVNicoVFs8+C6+/\nrv6KeqOwEBGiCJYtg717e/9e6q+oTwoLESGXCx3cjz7a+/eKY7jggjBDXOqHwkJEuOqqsOVpb5ui\nDhwIM7fVBFV/FBYiwumnw9SpvQ+LZcvCyCo1QdUfhYWIAKEpasWKsABgT8VxuEIpzAyX+qGwEBEg\ndHIfPgyLF/f8PfJ5mDYN3ve+8tUl1UFhISJAWPCvubnnTVE7doQrEzVB1SeFhYgAR1eI7emiggsW\nhC1a1bldnxQWIvKeXA6efBLeeuvkXxvHMHBgaIaS+qOwEJH3RFFYrmPhwpN/bRyHju1+/cpellQB\nhYWIvGfqVDj11JNvinrpJVi/Xk1Q9UxhISLvaW6GK688+U5uLfFR/xQWInKMKILVq+GNN0p/TT4P\n55wDEyakV5dkS2EhIseIonC/YEFpxx85AvPmhasKs9TKkowpLETkGFOmwBlnlN4U9dRT8Oab6q+o\ndwoLETlGU1PYEKnUsFB/RWNQWIjIcaIojG7atKn4sXEMF14I739/+nVJdhQWInKcQr9FsSG0e/eG\ntaTUBFX/FBYicpwLL4TBg4s3RS1dCvv3qwmqESgsROQ4ffqEpT/mzw8zursTx2HG9syZlatNsqGw\nEJEuRVHos9iwoftj8vmw+OBpp1WuLsmGwkJEupTLhfvumqLefDMsOqgmqMagsBCRLl1wAQwf3n1Y\nFJqo1LndGBQWItIls9AU1dbWdb9FHIcd8VpbK1+bVJ7CQkS6lcuFNaLWrj32efcQFlEU9tyW+qew\nEJFuFeZbdG6KevFFePllNUE1EoWFiHRr7FgYM+b4sIjjcK/O7cahsBCRE4qio/trF8QxnHsujB+f\nWVlSYQoLETmhXA527IBVq8Ljw4fDlcY112hJ8kaisBCRE+o836K9HXbtUhNUo0k1LMxsjpmtM7P1\nZnZnN8fcaGZrzGy1mf0see5iM1uWPPe0mX0kzTpFpHsjRoQ5F4VFBQtLks+enV1NUnlNab2xmfUF\n7gGuATYDK8zsYXdf0+GYFuAuYLq77zCzs5NfvQt83N1fMLNzgJVmNtfdd6ZVr4h0L5eDf/gHOHgw\n9FdMmQJDhmRdlVRSmlcWU4H17r7B3Q8A9wHXdzrmVuAed98B4O5bk/vn3f2F5OdXga3A0BRrFZET\niCLYswcWLYJHH9WQ2UaUZliMADpunbI5ea6j84HzzWypmS03szmd38TMpgLNwItd/O42M2s3s/Zt\n27aVsXQR6WjWrHD/9a+HqwuFReNJMyy6GifRedGAJqAFmAXcDPzAzM587w3MhgM/BT7p7kc6vRZ3\nv9fdW929dehQXXiIpGXoULjoIliyBPr3h+nTs65IKi3NsNgMjOrweCTwahfHPOTuB919I7COEB6Y\n2RnAb4G/dPflKdYpIiUozOa+6io45ZRsa5HKSzMsVgAtZjbWzJqBm4CHOx3zIJADMLMhhGapDcnx\nvwZ+4u6/SLFGESlRISw0ZLYxpRYW7n4IuB2YC6wF7nf31WZ2t5ldlxw2F9huZmuANuCL7r4duBGY\nAdxiZk8lt4vTqlVEivvgB+Ev/gJuuSXrSiQL5ifaM7GGtLa2ent7e9ZliIjUFDNb6e5FF5rXDG4R\nESlKYSEiIkUpLEREpCiFhYiIFKWwEBGRohQWIiJSlMJCRESKUliIiEhRdTMpz8y2AS/34i2GAG+W\nqZws1ct5gM6lWtXLudTLeUDvzuVcdy+6EmvdhEVvmVl7KbMYq129nAfoXKpVvZxLvZwHVOZc1Awl\nIiJFKSxERKQohcVR92ZdQJnUy3mAzqVa1cu51Mt5QAXORX0WIiJSlK4sRESkKIVFwsy+YWZPJxst\nPWJm52RdU0+Z2bfM7LnkfH7dcV/zWmNmHzaz1WZ2xMxqbuSKmc0xs3Vmtt7M7sy6nt4wsx+Z2VYz\nezbrWnrDzEaZWZuZrU3+bN2RdU09ZWYDzOxxM1uVnMtfpfZZaoYKzOwMd387+fnPgYnu/umMy+oR\nM/sgMN/dD5nZfwdw9y9nXFaPmNkE4Ajwd8AX3L1mdrgys77A88A1hP3mVwA3u/uaTAvrITObAewh\nbHd8Ydb19JSZDQeGu/sTZnY6sBL4o1r8/2JmBpzm7nvMrB+wBLjD3ZeX+7N0ZZEoBEXiNKBmU9Td\nH0m2tQVYDozMsp7ecPe17r4u6zp6aCqw3t03uPsB4D7g+oxr6jF3XwS8lXUdveXur7n7E8nPuwnb\nPo/Itqqe8WBP8rBfckvlu0th0YGZfdPMNgEfA/5L1vWUyZ8A/5J1EQ1qBLCpw+PN1OiXUr0yszHA\nFOCxbCvpOTPra2ZPAVuB2N1TOZeGCgszy5vZs13crgdw96+4+yjgH4Hbs632xIqdS3LMV4BDhPOp\nWqWcS42yLp6r2SvWemNmA4EHgM91almoKe5+2N0vJrQgTDWzVJoIm9J402rl7leXeOjPgN8CX0ux\nnF4pdi5m9gngD4HZXuUdUyfx/6XWbAZGdXg8Eng1o1qkg6R9/wHgH939V1nXUw7uvtPMFgBzgLIP\nQmioK4sTMbOWDg+vA57LqpbCCOWXAAAC0ElEQVTeMrM5wJeB69z93azraWArgBYzG2tmzcBNwMMZ\n19Twkk7hHwJr3f3bWdfTG2Y2tDDa0cxOAa4mpe8ujYZKmNkDwAWEkTcvA5929y3ZVtUzZrYe6A9s\nT55aXsMju24A/i8wFNgJPOXuf5BtVaUzs38D/G+gL/Ajd/9mxiX1mJn9HJhFWOH0DeBr7v7DTIvq\nATO7ElgMPEP4+w7wn939d9lV1TNm9vvA/yP8+eoD3O/ud6fyWQoLEREpRs1QIiJSlMJCRESKUliI\niEhRCgsRESlKYSEiIkUpLEROgpntKX7UCV//SzMbl/w80Mz+zsxeTFYMXWRm08ysOfm5oSbNSnVT\nWIhUiJlNAvq6+4bkqR8QFuZrcfdJwC3AkGTRwXnARzIpVKQLCguRHrDgW8kaVs+Y2UeS5/uY2feS\nK4XfmNnvzOzfJy/7GPBQctx5wDTgL939CECyOu1vk2MfTI4XqQq6zBXpmX8HXAxMJsxoXmFmi4Dp\nwBjgIuBswvLXP0peMx34efLzJMJs9MPdvP+zwKWpVC7SA7qyEOmZK4GfJyt+vgEsJHy5Xwn8wt2P\nuPvrQFuH1wwHtpXy5kmIHEg25xHJnMJCpGe6Wn78RM8D7AUGJD+vBiab2Yn+DvYH9vWgNpGyU1iI\n9Mwi4CPJxjNDgRnA44RtLf846bsYRlh4r2AtMB7A3V8E2oG/SlZBxcxaCnt4mNlgYJu7H6zUCYmc\niMJCpGd+DTwNrALmA19Kmp0eIOxj8Sxh3/DHgF3Ja37LseHxKeD9wHozewb4Pkf3u8gBNbcKqtQv\nrTorUmZmNtDd9yRXB48D09399WS/gbbkcXcd24X3+BVwVw3vPy51RqOhRMrvN8mGNM3AN5IrDtx9\nr5l9jbAP9yvdvTjZKOlBBYVUE11ZiIhIUeqzEBGRohQWIiJSlMJCRESKUliIiEhRCgsRESlKYSEi\nIkX9f2+mz9UXkoc4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f39ac05f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#需要调优的参数\n",
    "C_s = np.logspace(-3, 3, 7)# logspace(a,b,N)把10的a次方到10的b次方区间分成N份  \n",
    "#penalty_s = ['l1','l2']\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "#    for j, penalty in enumerate(penalty_s):\n",
    "    tmp = fit_grid_point_Linear(oneC, X_train, y_train, X_val, y_val)\n",
    "    accuracy_s.append(tmp)\n",
    "\n",
    "x_axis = np.log10(C_s)\n",
    "#for j, penalty in enumerate(penalty_s):\n",
    "plt.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "plt.savefig('SVM_RentListing.png' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最好应该是0.701，对应的C是1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5.2 RBF核SVM正则参数调优\n",
    "RBF核是SVM最常用的核函数。 RBF核SVM 的需要调整正则超参数包括C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和核函数的宽度gamma C越小，决策边界越平滑； gamma越小，决策边界越平滑。\n",
    "采用交叉验证，网格搜索步骤与Logistic回归正则参数处理类似。\n",
    "这里我们用校验集（X_val、y_val）来估计模型性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "def fit_grid_point_RBF(C, gamma, X_train, y_train, X_val, y_val):\n",
    "    # 在训练集是那个利用SVC训练\n",
    "    SVC3 =  SVC( C = C, kernel='rbf', gamma = gamma)\n",
    "    SVC3 = SVC3.fit(X_train, y_train)\n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC3.score(X_val, y_val)\n",
    "    \n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.698\n",
      "accuracy: 0.8015\n",
      "accuracy: 0.98\n",
      "accuracy: 0.997\n",
      "accuracy: 0.997\n",
      "accuracy: 0.997\n",
      "accuracy: 0.984\n",
      "accuracy: 0.9965\n",
      "accuracy: 0.997\n",
      "accuracy: 0.997\n",
      "accuracy: 0.9975\n",
      "accuracy: 0.9975\n",
      "accuracy: 0.9965\n",
      "accuracy: 0.997\n",
      "accuracy: 0.9975\n",
      "accuracy: 0.9975\n"
     ]
    }
   ],
   "source": [
    "#需要调优的参数\n",
    "C_s = np.logspace(-2, 2, 5)\n",
    "gamma_s = np.logspace(-2, 2, 5)  \n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j, gamma in enumerate(gamma_s):\n",
    "        tmp = fit_grid_point_RBF(oneC, gamma, X_train, y_train, X_val, y_val)\n",
    "        accuracy_s.append(tmp)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd4VHX2+PH3SQ8QAoGAQAghdAgQ\nQuirAjYEAUV2ERdFrFj267q7rr38WGWtq+6uHREsSxFBUBBFpKjUIL13SGiBQCCkTSaf3x8ziZOQ\nMknmZlLO63nmycxtc+YScube+7nniDEGpZRSqiQ+3g5AKaVU1afJQimlVKk0WSillCqVJgullFKl\n0mShlFKqVJoslFJKlUqThVJKqVJpslBKKVUqTRZKKaVK5eftADylcePGJioqytthKKVUtbJhw4bT\nxpjw0parMckiKiqKhIQEb4ehlFLViogcdmc5PQ2llFKqVJoslFJKlUqThVJKqVJZlixEZKqInBKR\nbcXMFxH5t4jsE5EtIhLnMm+8iOx1PsZbFaNSSin3WHlkMQ0YUsL864F2zse9wLsAIhIGPAf0AXoD\nz4lIQwvjVEopVQrLkoUxZiWQUsIiI4FPjMMaoIGINAOuA5YYY1KMMWeBJZScdJRSSlnMm9csWgBH\nXV4nOqcVN10ppZSXePM+Cylimilh+qUbELkXxyksIiMjPReZUh50/MABkvbsA6BAF2Pj8ottipnn\nnGbMb/8t8pczLj9MEdtzWabwesXNK3bbRcZUtm0XXu/SzysF37SoD1N4WlFBFAiyqOAKPTemiNcF\nn5sC71/ofUqKqcR1Sn7P/OdFbN9mN9jsdmw5udhycvGt78e4f96NlbyZLBKBli6vI4BjzukDC01f\nXtQGjDEfAB8AxMfHazNxVeVkpqfx9QsbsQXoZbcyMbkuL4r6/qhcBR5z6766CvFmslgAPCQiM3Fc\nzE41xhwXke+AyS4Xta8FnvBWkEpVxNIps7AFtKZh3U2EtgwB+e07jQgU+AoueT9Mgb+PkrdMgb+Z\nxrl+3kKm4GzntqXQ6/yn+TMKbcflvaXQtN+2VfB7WYHPkb+cy7f5nEzITEWyUiHzXP5DMlLBnllg\nH0hQCAQ3wPgF/faGUmDDjp/i/HQFlnH96YxCCs8ruI7JW7aY7Rhx2RmFtv3b+q5xSaH1C76nAWwm\nF5vdkGXPJdueS3aOwZab9zyXHGPy48o71eLv74O/nw8Bvj4E+Lv89Pch0M+XsAbWn1mxLFmIyAwc\nRwiNRSQRxwgnfwBjzHvAImAosA9IByY456WIyD+A9c5NTTLGlHShXKkq69SmbPxJ5eY37icwONjb\n4VjDGMg4C2f2Q8p+SDlQ8Hlm6m/Lig80agmN2kDY5RDWBsKiHa8btAK/AO99Dg+4kGnj2LlMjp3L\nIOlcBsfyH5kkncvgxPlM7LkFk239ID+aNwimRXgwzRvkPYJo4XzeJCQQP1/v3xJnWbIwxowtZb4B\nHixm3lRgqhVxKVVZDm7bSrp/O0IDt9SMRJGe4kwCBxyJwPV54YQQGuFIBF1/70gGYW2cCSES/AK9\n9xkqIMeey8kLWfkJIMklEeS9vpCZU2AdPx/hstAgmjcIpnfrMJo3CMpPCC0aBNMsNIiQIH8vfaKy\nqTGFBJWqalZP/wGkOz3G9PF2KO5LT3E5MiiUFDLPuSwo0KClIxHEjHYeKTiTQsNW1S4hGGM4n5nj\nciSQQZIzCeQ9TpzPpNBBAQ3r+NO8QTAtw+rQN7rRJcmgcb1AfH1qxjUXTRZKWcBut3PxdHOCzAG6\nDLB2lEqZ5SWEwqeLzuy/NCGEtoRG0RBz82+ni6phQsjOyeXk+cwCp4aSzmVyPPW3o4O0rIJHBQG+\nPjRrEETz0GD6tWlMC5dEkHeqqE5A7fkTWns+qVKV6KeZX5IdGE7zVme8E0B6CqQcLPoaQsZZlwWd\nCSGsNcSMKngNoWFUtUgIxhjOpdsKXiNILZgYTl3IKjiiFWhUN4DmDYJp3bguA9o2zr9GkJcIGtcN\nxKeGHBV4giYLpSxwaPlxfE1dBt37e+veJOMsnDlQ9DWESxJChCMJdLmp0DWEVuAfZF2MHpCVY+dE\n/h//zCKvGWTY7AXWCfTzcVwTaBDEFe3C808L5SWC5g2CCfL39dInqp40WSjlYclJiWTQgTq+u2jQ\naFjFNpZx1nlkcKDQUcIByHAdJJiXEFpD5xsLXUOIqvIJwdU3W46xaOvx/GsGyReyLlkmPCSQ5g2C\n6XBZCIM6NClwnaB5gyDC6gYgokcFnqTJQikPWzFlHrm+XWh7bbR7K2SccyaCg5deQ8goNGq8foTj\nGkLnEb8dHYRFOxNC9R9xtePYeR6euYmmIYG0aVKPTh2bXDKc9LLQIAL99KigsmmyUMrDUg+GECjH\n6Tvylt8mZpwrdFHZ5UghvdB1jfp5RwgjLr2GUAMSQnHsuYYn5m6hYZ0Avn34CkLrVI8hpbWFJgul\nPOjX738gMyiSxmFb8PX1ddyw9tnNsH9pwQXrt3AkgY43/DbCKCzakSRqcEIoySerD7E5MZV/j+2h\niaIK0mShlAdtm78FyY3hd3fe4Jhw6GdHoug5Adpe9ds1hIA6Xo2zqjl2LoPXvtvNwA7hDO/WzNvh\nqCJoslDKQzIupJGR2ZZg9tKi7bWOiQkfQVADGPLPWnvEUBpjDM/O30augX+MjNEL01WU9wuOKFVD\n/DBlBjn+9WjRJ8Qx4cJJ2Pk1xP5RE0UJFm87wQ87T/GXa9rTMkyPuKoqPbJQykOSt9jxl7MMHD/G\nMWHjp5CbA/ETvBtYFXY+08ZzC7bTpXl9JgyI8nY4qgR6ZKGUB+zbuJGMgLbUqX+YgMBAyLXDhmnQ\n+gpo3M7b4VVZryzexem0LF4a1a1KVFZVxdN/HaU8YN1ny0B8iLulv2PC3iWQehTi7/JuYFVYwqEU\nPltzhDsHtKZrRKi3w1Gl0GShVAXl2GxcTIkgKGs/nfv2dUxMmAr1mkLHCt7BXUNl5+TyxNyttGgQ\nzCPXtPd2OMoNmiyUqqCVM74kO7AxjTo5q5aePQx7v4e428FX7xcoygcr97P3VBov3BhD3UC9dFod\n6L+SUhV0ZOUpfKnH4HucF7Y3THO01+x5hzfDqrIOJKfx7x/3cUO3Zgzq2MTb4Sg36ZGFUhVw6ugR\nMqQ9wX57qd8wDHKyHaOg2g9xFPZTBRhjeHLeVoL8fHh2eGdvh6PKQJOFUhWw4sOvyPUNoMNQ54in\nXV/DxWSIv9O7gVVRX2xIZM2BFJ4Y2okmIdWnEq7SZKFUhZw/EkpgZhK9hl3vmLB+qqNHRJurvBtY\nFXQ6LYvJi3bSOyqMMfEtvR2OKiNNFkqV0/pvvyMzqCX1I1IcRQOTd8Phnx034fnof63CXvhmB+lZ\ndiaPitEOdNWQ/kYrVU47v96B5Nq4/M7hjgkJU8HHH2LHeTewKmjFnmS+2nSM+we2oW2TEG+Ho8pB\nk4VS5XAh9Rzp2W0Jtu+hWXQ0ZF+ETTOg80ioF+7t8KqUjGw7T3+1lejwujwwqI23w1HlpMlCqXJY\n9uEs7H51iejf0DFh21zISoVeesd2YW8u3cPRlAz+eVNX7XBXjVmaLERkiIjsFpF9IvJ4EfNbichS\nEdkiIstFJMJlnl1ENjkfC6yMU6myOr0d/LNTGHS7896KhI8gvBNE9vNuYFXM9mOpTPnpILf0akmf\n6EbeDkdVgGXJQkR8gbeB64HOwFgRKTyw+jXgE2NMN2AS8E+XeRnGmFjnY4RVcSpVVns2JJAR0Ia6\noUfw8/eHpF/h2EbHcFntxZDP0SZ1Kw3rBPDE9Z28HY6qICuPLHoD+4wxB4wx2cBMYGShZToDef0m\nlxUxX6kqZ/3nKwHoNe4Kx4SEqeBfB7qP8WJUVc8nqw+xJTGV54Z31japNYCVyaIFcNTldaJzmqvN\nwM3O5zcBISKSd6waJCIJIrJGRG60ME6l3JZjs3HxXEuCsw/Qvmc8ZJyDrXOg62gI0sqpeZLOZfDq\nd7sZ1CGcG7RNao1gZbIo6njcFHr9N+BKEdkIXAkkAc5qbEQaY+KBW4E3ReSSYRQicq8zoSQkJyd7\nMHSlirbsk1nYAhrRuIvzV3nLLMjJ0FLkLowxPPvVNoyBSdomtcawMlkkAq63aUYAx1wXMMYcM8aM\nMsb0AJ5yTkvNm+f8eQBYDvQo/AbGmA+MMfHGmPjwcB2uqKyXtPosvjkXGXTPGDAG1n8ELXpC81hv\nh1ZlfLvtBEt3neKv12qb1JrEymSxHmgnIq1FJAC4BSgwqklEGotIXgxPAFOd0xuKSGDeMsAAYIeF\nsSpVqhOHDpLu0546AfsICW0Ah3+B07u1DpSL1Awbzy/YTkyL+tzRP8rb4SgPsixZGGNygIeA74Cd\nwGxjzHYRmSQieaObBgK7RWQP0BR40Tm9E5AgIptxXPh+yRijyUJ51copCzA+/nS6wTmoL2Gq4zpF\nl1HeDawK0TapNZel/SyMMYuARYWmPevyfA4wp4j1VgFdrYxNqbI6nxhGIEfpNXQ8pJ2CHQug190Q\noKdawNEm9fO1R7j7d62JaaEX+2saTf1KuWHt1wvJCmpBaGSqY8LGTyHXpqegnLRNas2nyUIpN+xa\nuAfJtXHFPTdCrt3RDS/qcgjXP4wA76/QNqk1nSYLpUpx/mwKGTntCM7dTdOWkbBvKZw7okcVTvuT\n0/iPtkmt8TRZKFWKHz+chd2vDpGXO4dnJ3wEdZtAxxu8G1gVYIzhqXlbCfLXNqk1nSYLpUqRssOP\ngKzTXHnraMcRxZ7vIO528Avwdmhel9cm9Ultk1rjabJQqgQ71qwhI6gNdcOSHEUDN0x3zOg53ruB\nVQGn07J4caGjTeoftE1qjafJQqkSbJy5CkwuvccNhJxs+PUTaH8dNIj0dmhe949vdpCRrW1SawtN\nFkoVIzsri4vnWxGcvY+2PXrA7oVw8ZTWgcLRJnX+pmM8MEjbpNYWmiyUKsby6bOwBTQkvJuzu9v6\njyA0Etpe5d3AvCw9O4en5m2lTXhd7h+obVJrC00WShUjae15/GxpXHX3GEjeA4d+gvg7wKd2twZ9\n64e9JJ7NYLK2Sa1VNFkoVYRj+/eR4due4KB91AmpDxs+Bh9/6HGbt0Pzqu3HUpny80HG9tY2qbWN\nJgulivDTR99gfPzoMrIbZKfDps+h03CoV3tvOnNtk/r4EG2TWttoslCqELvdzoVj4QRlHqHntVfD\n9rmQmQq9aveF7emrtE1qbabJQqlCHEUDmxEadd4xIWEqNO4ArQZ4NzAvSjqXwWvfa5vU2kyThVKF\n7F18AB97NlfeMwqObYKkDY46ULW0Pahrm9R/3KhtUmsrTRZKuUg9c9pRNJA9hLeIcNSB8q8D3W/x\ndmhe49omNaKh9u6orTRZKOVi2YezsfsF0+rKpo7rFFvnQMzNENzA26F5RWqGjee0TapCk4VSBZzZ\nFUBAVjJXjB0Nm2eBLb1WlyJ/efEuzmibVIUmC6Xybf/lFzKDoqnb+Bi+Pj6OU1DNe0CLOG+H5hXr\nD6Xwv7VHuEvbpCo0WSiVb+PstWBy6XvbYDiyGpJ31do6UFk5dm2TqgrQZKEUkJWRQfqFKOrY9hLd\nrbujDlRgKMSM8nZoXvH+igPsc7ZJrROgbVKVJgulAFg+fTa2gAY0iQ2AtGTYMR9ix0JAXW+HVun2\nJ6fx3x/3Mbx7c22TqvJpslAKOLYuDT/bBQbfNQY2fQa5tlp5YdsYw5NznW1Sb9A2qeo3mixUrXd0\nz24y/NpTJ3g/wcF1IOFjaPU7CO/g7dAq3RcJiaw96GiTGh4S6O1wVBViabIQkSEisltE9onI40XM\nbyUiS0Vki4gsF5EIl3njRWSv86E9LJVlfvl4EcbHl66jesD+pXDuMPSqfUcVp9OyeHHRTnq31jap\n6lKWJQsR8QXeBq4HOgNjRaTwce1rwCfGmG7AJOCfznXDgOeAPkBv4DkRaWhVrKr2stvtpJ1oSlDm\nYWIHD3LUgaobDh2Hezu0SpffJvWmrtomVV3CyiOL3sA+Y8wBY0w2MBMYWWiZzsBS5/NlLvOvA5YY\nY1KMMWeBJcAQC2NVtdTqeQvICryMBtEX4dxR2LPY0bPCL8DboVWq5btPubRJreftcFQVZGWyaAEc\ndXmd6JzmajNws/P5TUCIiDRyc12lKmz/ksP42LMYdN/N8Ot0MAZ63uHtsCpVenYOT3+1TdukqhJZ\nmSyKOo41hV7/DbhSRDYCVwJJQI6b6yIi94pIgogkJCcnVzReVcucPXWS9Nz2BMsewho3hl8/gXbX\nQsNW3g6tUr3pbJP6z1HdtE2qKpaVySIRcL1KFgEcc13AGHPMGDPKGNMDeMo5LdWddZ3LfmCMiTfG\nxIeHh3s6flXDLftwDrm+QURfFQG7FkLayVo3XHZbUiofOduk9m4d5u1wVBVmZbJYD7QTkdYiEgDc\nAixwXUBEGotIXgxPAFOdz78DrhWRhs4L29c6pynlMWf3BhGQdZIBo2901IEKjYR213g7rEqjbVJV\nWViWLIwxOcBDOP7I7wRmG2O2i8gkERnhXGwgsFtE9gBNgRed66YA/8CRcNYDk5zTlPKIzStWkhnU\nmnpNTuB79gAcXAk9x4NP7TkNM23VIbYmpfL8CG2TqkpnadEXY8wiYFGhac+6PJ8DzClm3an8dqSh\nlEdtmZMApisD7rjOcROej59jFFQtkXQug9e/383gjk0Y1lXbpKrS6R3cqtbJTE8jPT2aOra9RLZt\nDZs+h07DIaSpt0OrFMYYnnG2SZ00sou2SVVu0WShap1lH88mx78+l/UMgu3zIPNcrSpFvmjrCX7U\nNqmqjDRZqFrnxIZM/GznGXznLY5S5I3bQ9TvvB1WpUjNsPH819vp2iJU26SqMtFkoWqVIzt3kO7f\njjp1DhB4bi8kJTiGy9aSUzF5bVL/OaqrtklVZaK/LapW+WXadyC+dBsd76gD5RcM3W/xdliVQtuk\nqorQZKFqDbvdTtqpZgRlHqR7n1jY8gXE3AzBNb9GpbZJVRWlyULVGr/Mnkd2YBMatsuELbPAdrHW\nlCLPb5N6k7ZJVeWjyULVGgeWJeFjz2TQ3Tc7TkE1i4UWPb0dluUKtEntoG1SVfloslC1QsrJ42SY\n9tTx2UPDzINwaketqAOV6yzpoW1SVUW5lSxE5EsRGeZSx0mpamXZ+1+S6xtIm2uiHHWgAkOh62hv\nh2W5LzYcZd3BFJ4apm1SVcW4+8f/XeBWYK+IvCQiHS2MSSmPO3egDoFZJ+h3bT/YMd8xAiqgrrfD\nslTyhSwmL9qlbVKVR7iVLIwxPxhj/gjEAYeAJSKySkQmiIhWIFNV2qYfl5EZFEW9y07iu2UG2LMh\nfoK3w7Kca5tULemhKsrt00rODnZ3AHcDG4G3cCSPJZZEppSHbJ27Ecm1M2D8ENjwMbQaAE1qdknu\n5btPsWDzMR4c1FbbpCqPcGsMnYjMBToCnwLDjTHHnbNmiUiCVcEpVVGZ6WmkZ7Qh2OyhpZ/A2UMw\n+Blvh2WpvDapbZvUY+LAaG+Ho2oIdwdc/9cY82NRM4wx8R6MRymPWjplFjn+rYmKrQfrp0Kdxo4K\nszVYXpvULyb20zapymPcPQ3VSUQa5L1wdrB7wKKYlPKYU5uy8c8+x8Abfwd7voW428Cv5o4K2paU\nypSfDjC2dyS9orRNqvIcd5PFPcaYc3kvjDFngXusCUkpzziwZbOjaGDIIQJ3zgJjoOcd3g7LMjn2\nXJ6Yu5VG9QJ5/HodsKg8y91k4SMuwylExBcIsCYkpTxjzac/gvjQY3Q8bJgOba+GhlHeDssy01cf\nZmtSKs8N70xosA5SVJ7l7jWL74DZIvIeYICJwGLLolKqgux2OxdPNyfIHKBLo3BIOwG93vR2WJZJ\nPJuubVKVpdxNFo8B9wH3AwJ8D0yxKiilKmrljDlkB4bTIuqMow5U/Qhod623w7KEMYZn528HtE2q\nso5bycIYk4vjLu53rQ1HKc84vOIEvqYeg37fFz59AgY9DT41c2TQwq3H+XHXKZ4e1knbpCrLuHuf\nRTvgn0BnIChvujFGB3GrKic5KZEMOlDHbxeh+1aBjx/E3e7tsCyRmm7j+QU7tE2qspy7F7g/xnFU\nkQMMAj7BcYOeUlXOig/nkusbQLtrImHT59DxBghp6u2wLPHS4l2cTc/WNqnKcu7+dgUbY5YCYow5\nbIx5HhhsXVhKlV/qofoEZh6nT1QGZJytsaXI1x1MYcY6bZOqKoe7ySLTWZ58r4g8JCI3AaV2URGR\nISKyW0T2icjjRcyPFJFlIrJRRLaIyFDn9CgRyRCRTc7He2X6VKrW2vDd92QGRRLSPBnfjdOgUTto\nfYW3w/K4rBw7T85ztEn989XtvB2OqgXcTRZ/BuoA/wf0BMYB40tawXkvxtvA9TiudYwVkcLdV54G\nZhtjegC3AO+4zNtvjIl1Pia6Gaeq5bYv2Ibk5nD5jR0hcZ3jqKIGjg56b7m2SVWVq9TfMucf/T8Y\nYx4F0gB3azv3BvYZYw44tzMTGAnscFnGAPWdz0OBY25uW6lLpF84T0ZmW4LZQ/OT+8AvyNG3oobZ\ndyqNt5ftY4S2SVWVqNQjC2OMHegpZR+83QI46vI60TnN1fPAOBFJBBYBf3KZ19p5emqFiFxexvdW\ntZCjaGA9WvQKhq1fQMzNUKdm1UfKzTU8Oc/RJvUZbZOqKpG7x68bgfki8gVwMW+iMWZuCesUlVxM\noddjgWnGmNdFpB/wqYjEAMeBSGPMGRHpCXwlIl2MMecLvIHIvcC9AJGRkW5+FFVTJW+x4y8pDOxm\ng+/TIP4ub4fkcXltUl++uau2SVWVyt1kEQacoeAIKAOUlCwSAddejhFceprpLmAIgDFmtYgEAY2N\nMaeALOf0DSKyH2gPFOidYYz5APgAID4+vnAiUrXIvo0byQhoS4PgLQRsmgOXdYMWcd4Oy6OSL2Tx\n4sKd9NE2qcoL3L2Duzw9KNcD7USkNZCE4wL2rYWWOQJcBUwTkU44bvhLFpFwIMUYYxeRaKAdcKAc\nMahaYt1ny0Biibu2KazbDsPfqnEXtid9s4NMWy6TR2mbVFX53L2D+2MuPYWEMabYAezGmBwReQhH\nEUJfYKoxZruITAISjDELgL8CH4rII87t32GMMSJyBTBJRHIAOzDRGJNS1g+naoccm42LKREEm/10\nylwPASEQM9rbYXnUst2n+HrzMR65uj1twrVNqqp87p6G+sbleRBwE26MXDLGLMJx4dp12rMuz3cA\nA4pY70vgSzdjU7Xciv/NITuwKS1anYDt8yBuPATWnD+o6dk5PD1P26Qq73L3NFSBP9wiMgP4wZKI\nlCqjIz8l40sIg3tmwarsGnfH9htL9pB0TtukKu8qbzGZdoAOP1Jed/LIITJ8OhDst5f6Oz+HyH7Q\ntOYMKd2WlMpHPx/UNqnK69y9ZnGBgtcsTuDocaGUV62csgDjE0PHfr5w8CAMesrbIXmMtklVVYm7\np6FCrA5EqfJIPRJKIEn0CVwLdRpB5xHeDsljpq06xNakVN6+NU7bpCqvc+s0lIjcJCKhLq8biMiN\n1oWlVOnWLVxMVlBL6jc/Cbu/hR7jwK9m3KjmaJO6h6s6NmFo18u8HY5Sbl+zeM4Yk5r3whhzDnjO\nmpCUcs+uhTuRXBtX9EgFkws9y3M7UNWT1yZVBCbdGKP3VKgqwd1kUdRyWupSec2F1HOkZ7eljn0P\nlx35AtpeBWGtvR2WR+S1Sf3rtR1o0SDY2+EoBbifLBJE5F8i0kZEokXkDWCDlYEpVZJlH87C7leX\niC5pcOF4jRkum9cmtVuEtklVVYu7yeJPQDYwC5gNZAAPWhWUUqU5vR38s88wsOFqqN8C2l3n7ZA8\n4qXFOzmbns3km7ri66Onn1TV4e5oqIvAJZ3ulPKGPRsSyAhoQ4OgX/E7vMIxXNa3+p8VdbRJPcq9\nV0Rrm1RV5bg7GmqJiDRwed1QRL6zLiylirf+s5UA9O5+EsQXetzm5YgqLivHzhNztxDRUNukqqrJ\n3a9jjZ0joAAwxpwVEW3RpSpdjs3GxdRIgs1+2iXPhY7DoH4zb4dVYe8u38/+5ItMm9BL26SqKsnd\naxa5IpJf3kNEoiiiCq1SVlv2ySxsAWE0jjwOGSnQq/o3ONp3Ko13lu1nRPfmDNQ2qaqKcvcrzFPA\nzyKywvn6Cpwd6pSqTImrzuJLKFdfthpsbSDqCm+HVCG5uYYn524lOMBX26SqKs2tIwtjzGIgHtiN\nY0TUX3GMiFKq0pw4dJAM3/bU8d9FnZNrHMNlfcpbC7NqmJ1wlHWHUnhqaCdtk6qqNHcLCd4NPIyj\nNeomoC+wmoJtVpWylKNoYFc6tTsE6YEQW7jxYvVy6kImkxc52qT+Pj7C2+EoVSJ3v5Y9DPQCDhtj\nBgE9gGTLolKqELvdzvnEMIIyj9Ir6yuIGQV1qnfJ7n98s1PbpKpqw91kkWmMyQQQkUBjzC6gg3Vh\nKVVQwqLFZAW1oH7jfZCdBvHV+8L2sl2ONqkPDW6rbVJVteDuBe5E530WXwFLROQsbrRVVcpTdi3c\ng4+9E1c2+xnqdoWIeG+HVG4Xs3J4+qtttGtSj4lXtvF2OEq5xd07uG9yPn1eRJYBocBiy6JSysX5\nsylk5LQj2OyiyYVf4co3oBqftnFtkxrgV70v0Kvao8x3/xhjVpS+lFKe8+OHs7D7dSCy6W4ICIGu\nf/B2SOW2LSmVqb8c5NY+2iZVVS96q6iq8lJ2+BEgp7nCZy50HweB1fMcf449l8fnbqFRvUAeG6Jt\nUlX1oslCVWk71qwhI6gNDf1W4Gcyq3Up8mmrDrEt6bxbbVJtNhuJiYlkZmZWUnSqpgsKCiIiIgJ/\n//K16NVkoaq0X2euAtONPk2XQfO+0LSLt0Mql6MpZWuTmpiYSEhICFFRUTqsVlWYMYYzZ86QmJhI\n69blaxJm6dU1ERkiIrtFZJ+IXFLiXEQiRWSZiGwUkS0iMtRl3hPO9XaLSM1oVqDKJDsri/TzrQjO\n2ksb++ZqWwfKGMMz87eVqU1qZmYmjRo10kShPEJEaNSoUYWOVC1LFiLiC7wNXA90BsaKSOHiN08D\ns40xPYBbgHec63Z2vu4CDAFF8fdIAAAgAElEQVTecW5P1SLLp8/CFtCQ8PAtEBwGnUZ4O6Ry+WbL\ncZbvTuZvZWyTqolCeVJFf5+sPLLoDewzxhwwxmQDM4GRhZYxQH3n81B+u3djJDDTGJNljDkI7HNu\nT9UiSWvP42e7wNVBX0KPceAf5O2QyizhUApPzdtKt4hQxlfTNql9+vQhNjaWyMhIwsPDiY2NJTY2\nlkOHDpVpO3PnzmXXrl1lfv/f/e53bNq0qczr5Xnttdf43//+V+71K8Pvf/97Dhw4UOS8xYsXExcX\nR9euXenZsyfLly8vcrkzZ85w1VVX0a5dO6677jpSU1M9GqOVyaIFcNTldaJzmqvngXEikggswtG+\n1d11VQ2WuHcPGb7tCfbZSrBvFsRP8HZIZfb99hP8ccpaGtUL5O1b46ptm9S1a9eyadMmJk2axJgx\nY9i0aRObNm0iKiqqTNspb7KoCJvNxqeffsqYMWMq9X3LauLEibz66qtFzmvSpAkLFy5k69atTJ06\nldtuK7rZ14svvsj111/P3r17ufzyy3nllVc8GqOVyaKo/xmFe2CMBaYZYyKAocCnIuLj5rqIyL0i\nkiAiCcnJWqqqJvnl40UYHz9iwn+ENoMhLNrbIZXJ/9YeYeJnG+jYrD5zJvajZVgdb4dkiW+//ZZ+\n/foRFxfHmDFjuHjxIgCPPvoonTt3plu3bjz22GP89NNPLFq0iEceeaRcRyV5PvvsM7p27UpMTAxP\nPvlk/vT333+f9u3bM3DgQO6++27+/Oc/A7BkyRJ69eqFr6/jLPaaNWvo1q0b/fv359FHHyU2NhaA\n/fv3c/nll9OjRw969uzJ2rVrAfjhhx8YNGgQo0ePpl27djz99NN88skn9OrVi27duuV/jnHjxvHg\ngw8yaNAg2rRpw8qVKxk/fjwdO3bkrrt+u9Z27733Eh8fT5cuXZg0aVL+9IEDB7J48WLsdvslnzku\nLo5mzRwNvrp27UpaWho2m+2S5ebPn8/48eMBGD9+PF999VW59nFxrBwNlQi0dHkdwaUlQu7CcU0C\nY8xqEQkCGru5LsaYD4APAOLj47UZUw1ht9u5cCycIA4TF7AB4j/3dkhuM8bw5g97eWvpXgZ1COft\nP8ZVuPPd//t6OzuOnfdQhA6dm9fnueEVG1l26tQpXnrpJZYuXUqdOnV48cUXeeutt7jrrrtYtGgR\n27dvR0Q4d+4cDRo0YOjQoYwePZobb7yxXO+XmJjI008/TUJCAqGhoVx99dV88803dO/enZdeeolf\nf/2VunXrMnDgQHr3dpy1/uWXX+jZs2f+NiZMmMD06dPp3bs3f/vb3/KnN2vWjCVLlhAUFMSuXbsY\nP358fsLYvHkzO3fuJDQ0lKioKB544AHWr1/P66+/zn//+19ee+01AFJTU1m2bBlffvklw4cPZ/Xq\n1XTs2JG4uDi2bdtGTEwML730EmFhYeTk5OQnoc6dO+Pr60tUVBTbtm2je/fuxe6D2bNn06dPnyKH\nv545c4bw8HAAWrRowfHjx8u1n4tj5ZHFeqCdiLQWkQAcF6wXFFrmCHAVgIh0AoJwVLNdANwiIoEi\n0hpoB6yzMFZVhayZv5CsoGaEhvwKIc2h/RBvh+SWHHsuT87byltL9zK6ZwQf3B5fo1ukrlq1ih07\ndtC/f39iY2P5/PPPOXToEGFhYfj4+HDPPfcwb9486tat65H3W7t2LYMHD6Zx48b4+/tz6623snLl\nyvzpDRs2JCAggNGjR+evc/z48fw/oKdPnyY7Ozs/kdx6628l7rOysrjrrruIiYnhlltuYceOHfnz\n+vTpQ9OmTQkKCiI6OprrrnMMzuzatWuBI6Thw4fnT2/evDmdO3fGx8eHzp075y83Y8YM4uLiiIuL\nY+fOnQXep0mTJhw7VnzJva1bt/L000/z7rvvurW/PD1AwrLfZGNMjog8BHwH+AJTjTHbRWQSkGCM\nWYCjidKHIvIIjtNMdxhjDLBdRGYDO4Ac4EFjzKXHZ6pG2vf9AXzsHRkYNhd6/gl8q/4f3IxsO3+a\nsZEfdp7koUFt+eu17T32n7WiRwBWMcYwZMgQPv3000vmJSQksGTJEmbOnMm7777L999/X+x2XP+A\njxo1imeffbbY9yvLdIDg4OD84aIlLff666/TsmVLPvvsM2w2G/Xq/VYlIDDwt6ZUPj4++a99fHzI\nycm5ZDnXZVyX27t3L2+99Rbr1q2jQYMGjBs3rsBQ1szMTIKDg5kzZw4vvPACANOmTSM2NpYjR44w\natQoPvvss2Lvk2jUqBHJycmEh4eTlJTEZZeVfj9PWVh6n4UxZpExpr0xpo0x5kXntGediQJjzA5j\nzABjTHdjTKwx5nuXdV90rtfBGPOtlXGqqiP1zGlH0UD7VhoHZkDc7d4OqVTn0rMZ99Falu46yT9G\nduFv13WoFcNe+/fvz4oVK/JH8Vy8eJG9e/dy4cIFzp8/zw033MAbb7zBxo0bAQgJCeHChQuXbCcg\nICD/onlxiQKgb9++LFu2jDNnzpCTk8PMmTO58sor6dOnD8uWLePcuXPYbDbmzp2bv06nTp3Yt28f\nAOHh4fj7+5OQkADAzJkz85dLTU2lWbNmiAjTp08vMbGU1/nz5wkJCaF+/focP36c7777rsD8vXv3\n0qVLF0aPHp2/P2JjYzl79izDhg3jtddeo2/fvsVuf8SIEUyfPh2A6dOnM3Jk4cGnFaMlL1WV8uMH\ns7H7BRMV+iN0HAr1m3s7pBIlnctg9Hur2ZqUyju3xnFbvyhvh1RpmjZtykcffcSYMWPo3r07/fv3\nZ8+ePaSmpjJs2DC6d+/O4MGD+de//gXA2LFjmTx5crkvcEdERDBp0iQGDhxIbGwsffv2ZdiwYURG\nRvLoo4/Su3dvrr32Wrp06UJoaCgAQ4cOZcWK32qfTp06lQkTJtC/f398fHzyl3vooYeYMmUKffv2\n5fDhwwWODDwlLi6Ozp07ExMTwz333MOAAQPy5x07dozQ0ND8U2au3nrrLQ4ePMhzzz2XP2z5zJkz\ngOMaTN6w4ieffJKFCxfSrl07Vq5cyaOPPurR+MWKDOoN8fHxJu8bg6q+PrpjCrlSjztb3ofv+HmO\nkVBV1K4T5xk/dR3p2Xam3B5Pn+hGHtv2zp076dSpk8e2V9OlpaVRr149bDYbI0eO5P7778+/hjBi\nxAjefPNNoqOj85cDx1DTlJQUXn/9dW+GDsCrr75KkyZN8kczWaWo3ysR2WCMKbVBjB5ZqCpj+y+/\nkBkUTV3/BHwbR0Prgd4OqVhrDpzh9++tRhC+mNjPo4lCld0zzzxDjx496NatGx06dOCGG27In/fy\nyy/nXzhesGABsbGxxMTEsHr1ap544glvhVxAo0aNGDdunLfDKFHVv3Koao2Ns9aC6Ua/hvMh/v/A\np2p+l1m09Th/nrmJyEZ1mH5n7zKV8FDWeOONN4qd5/pN+tZbby0wCqqquPPOql9NWZOFqhKyMjJI\nT2tNsH0nreulQuwfvR1SkaavOsTzX28nLrIhH42Pp0GdAG+HpFSl0GShqoTl02djC2hJC7/PoctN\nUKdqdZEzxvDa97t5e9l+runclP+M7UGQv9a2VLWHJgtVJRxbl4afnOeq8B+hV9Vq726z5/LE3K3M\n2ZDI2N6R/GNkF/x8q+YpMqWsoslCed2RXTvJ8GtPSM5ygpp3gohe3g4pX3p2Dg98/ivLdyfzyNXt\n+b+r2taKeyiUKky/HimvWzV9McbHl64NFzraplaRP8Zn0rIY++FaVu5J5p+juvLw1e1qZaLQEuXW\nK6lE+alTpxg4cCB169bNL5BYlOpcolypUtntdtJONCUo4yCxjZKh2x+8HRIAR86kM/q91ew6fp73\nb4tnbO9Ib4fkNVqi3HollSjPK9L48ssvl7iN6lyiXKlSrfpyPlmBl9EgaIUjUQSGeDsktiWlMurd\nVaRczOZ/9/Thms5NvR1SlaUlyh2fw8oS5fXq1WPAgAEEBZXc/Ks6lyhXqlQHlh7BJzeYQeFLIH6J\nt8Ph572nmfjZBkKD/Zl5bx/aNvF+8uLbx+HEVs9u87KucP1LFdqEliiv/BLlJanOJcqVKtHZUydJ\nz21PsO1Xwtp0hctivBrP/E1JTJi2joiGwXx5f/+qkSiqMC1RXrklysuq2pQoV6o0yz6cQ65vJ6KD\nv4f4+70ay5SfDvDCwp30aR3GB7fHExp8aXMZr6ngEYBVtER55ZUod0e1LlGuVEnO7g0iIPMEA5od\ngc7lOzVRUbm5hsmLdvLCwp0M7XoZ0+/sXbUSRRWmJcrLprwlyt1ldYlyPbJQXrF5xUoyg1oTZvsC\n355/BP+SL95ZITsnl7/P2cxXm45xe79WPDe8C74+tW9obHm5lijPzs4GYPLkyQQHBzNq1CiysrLI\nzc0tUKL8vvvu4/XXX+err74q82gq1xLlxhiGDx/OsGHDAPJLlLdo0eKSEuWuF5jzSpSHhIRwxRVX\nFChRPnr0aGbMmMHVV19teYny6Ohot0uU53329PR0bDYbc+bMYenSpXTo0IEJEybw8MMPExsby5NP\nPskf/vAH3n//fVq3bs2sWbM8Gr+WKFde8emf/sX57K4MbziRyEe/h0ZtKvX907JymPjpBn7ed5pH\nr+vAAwPbVKl7KLREedloiXL3aIlyVa1kpqeRnh5NnYxtRHbuWumJIvlCFrd8sJrVB87w6uhuPDhI\n78qu7rREufX0NJSqdMs+nk2OfxSRgT9Cr3sr9b0Pnr7I+KnrSL6QxZTx8Qzq0KRS319ZQ0uUW0+T\nhap0JzZk4kcqgzvuh/bXV9r7bj56jjunrccAM+7tS2zLBpX23kpVd3oaSlWqQ9u3ke7fjjq5qwjs\ndRv4Vs73leW7TzH2wzUEB/gyZ2I/TRRKlZEmC1WpVk//HsSX2MZLoKe1F/PyfLkhkbunJxDVqC5z\nH+hPdHi90ldSShWgp6FUpbHb7aQlNycodx9dB3aB+s0tfT9jDO+tOMDLi3cxoG0j3hvXk5AgvYdC\nqfLQIwtVaX6ZPY/swCY0DFzuKEVuodxcw//7egcvL97FiO7N+fiO3pooyklLlFuvcIny9evXExMT\nQ9u2bXnkkUeKXMcYwwMPPEDbtm3p3r17hfaROyxNFiIyRER2i8g+EXm8iPlviMgm52OPiJxzmWd3\nmbfAyjhV5TjwYxK+ORkMarsHogdZ9j5ZOXb+NHMj01Yd4u7ftebNMbEE+On3ovLSEuXWK1yifOLE\niXz88cfs3buX7du3s2TJpUU2v/76a44ePcq+fft4++23efDBBy2N0bL/QSLiC7wNXA90BsaKSGfX\nZYwxjxhjYo0xscB/gLkuszPy5hljRlgVp6ocKSePk0F7gm0JNBxwG/hY86t3PtPG+KnrWLjlOE8N\n7cTTN3TGR+/KtoyWKHd8Dk+WKD969CiZmZn06tULEeG2224rstz4/Pnzuf322wHH0deJEydITk4u\n1351h5XXLHoD+4wxBwBEZCYwEthRzPJjgecsjEd50bL3vyTXtzNtw5ZB7JeWvMfJ85mMn7qO/clp\nvDkmlht7tLDkfSrby+teZleKZ7+RdwzryGO9H6vQNrREuTUlyjMyMmjZsmV+bBERESQlJV2yP5KS\nkopcrriSIRVl5bF5C+Coy+tE57RLiEgroDXwo8vkIBFJEJE1IuKdKnPKY84dqEtg5nH69msDdRt5\nfPv7k9MY9c4qjqakM/WOXjUmUVRlWqLcmhLlRZVgKqrCgLvLeYqVRxZFRV1cIapbgDnGGNc2UZHG\nmGMiEg38KCJbjTH7C7yByL3AvQCRkbW37WVVt+nHH8kMakUj2yx8+97j8e1vOHyWu6avx89HmHVf\nP2JahHr8PbypokcAVtES5daUKI+IiODo0d++ZycmJtK8+aUjB/OW69u3b4nLeYqVRxaJQEuX1xFA\ncZ09bgFmuE4wxhxz/jwALAd6FF7JGPOBMSbeGBNv1aGXqritczchuXZ+134PtOzj0W0v3XmSP05Z\nQ4Ngf768v3+NSxRVmZYoLxt3S5S3bNmSwMBA1q9fjzGGTz/9tMhy4yNGjOCTTz4B4Oeff6Zp06aW\nnYICa48s1gPtRKQ1kIQjIVxSlEVEOgANgdUu0xoC6caYLBFpDAwAPNt9XFWKjAtppGe0Idi2hYhB\nt4IHD5NnrT/Ck/O2EdO8Ph/d0YvG9TxfVloVT0uUl01ZSpS/++673HHHHWRmZnLDDTdwzTXXAPD2\n228TGBjI3XffzfDhw/n2229p06YNdevWze9lYRljjGUPYCiwB9gPPOWcNgkY4bLM88BLhdbrD2wF\nNjt/3lXae/Xs2dOoqufrNz80/71vqVn84HBjMlI9ss3c3Fzz7x/2mFaPfWNu/2itScu0eWS7VcmO\nHTu8HUK1cuHCBWOMMdnZ2eb66683CxYsyJ83fPhws3///gLLGWPMCy+8YP7yl79UbqDFeOWVV8y0\nadMsf5+ifq+ABOPG33NL7+A2xiwCFhWa9myh188Xsd4qoKuVsanKkbzJhr+cZdB1URBUv8Lbs+ca\nnp2/jc/XHmFUXAtevrkb/r56D0Vt98wzz7B8+XIyMzMZMmRIkSXKo6OjWbBgAa+88go5OTlERUUx\nbdo07wXtojqUKNfmR8oyB7Zs5tu3kwnN/pZxk8dDs24V2l6mzc7DMzfy3faT3D+wDX+/rkON7UOh\nzY+UFSrS/EhrQynLrPlkKUgsce33VDhRpKbbuPuT9SQcPstzwzszYUBrD0WplHKHJgtlCbvdzsUz\nLQi276bz9RVrNnPsXAbjp67j8Jl0/jO2Bzd0s7YAoVLqUposlCVWzphDdmA4zeVr6PJhubez5+QF\nbv9oHRezcph+Z2/6tfH8DX1KqdJpslCWOLz8OL7UY/CwZuAfXK5trDuYwt3T1xPk78vsif3o1Kzi\nF8iVUuWjw0iUxyUnJZIhHQm2rSP08vvKtY3F204w7qO1NA4JZO4D/TVReJGWKLde4RLljz/+OBER\nETRoUHJHxxdeeIG2bdvSsWNHfvjhB0tj1GShPG7Fh3PJ9Q2gffQeaNy2zOt/tuYwD3y+gS7N6/Pl\nxP5ENKxjQZTKXVqi3HqFS5SPHDmSNWvWlLjOli1bmDt3Ljt27GDhwoXcf//95ObmWhajJgvlcamH\nQgjMTKL3qD+UaT1jDK9/v5unv9rGoA5N+N/dfWlYN8CiKJUnaIlyx+fwZIlygH79+nHZZZeVuC/m\nz5/P2LFjCQgIoE2bNkRGRrJhw4Zy7Vd36DUL5VEbvvuezKBWNM79At/O/3Z7vRx7Lk/O28rshETG\nxLfkxZti8NOb7QA4MXkyWTs9+408sFNHLnP5Y1seWqLcmhLl3bt3d2t/JCUlMXDgwPzXeSXKe/Xq\nVa79Wxr936g8avtXW5BcG1dc3Qh83WtjmpFt575PNzA7IZH/G9yWl27uqomiGtAS5daUKHdXUTdU\nV9cS5aqWSb9wnozs9gTbttDsuolurZNyMZu7pq9n89FzvHBjDOP6trI4yuqnokcAVjFaotySEuXu\ncreUuafo1zflMUs/+B85/vWIiNgNoRGlLn80JZ3R761i+7HzvPPHnpooqhktUV427pYod9eIESOY\nMWMG2dnZ7N+/n8OHDxc45eZpmiyUxyRvzcU/O4Ur/zi61GV3HDvPze+u4vSFLD6/uw9DYkq+mKeq\nHtcS5d27d6d///7s2bOH1NRUhg0bRvfu3Rk8eHCBEuWTJ08u9wVu1xLlsbGx9O3bl2HDhhEZGZlf\novzaa6+9pET5ihUr8reRV6K8f//++Pj4FChRPmXKFPr27cvhw4ctL1F+zz33lFii/C9/+QtRUVGc\nP3+eiIgIXnjhBQDmzZuXf2G8e/fu3HjjjXTq1ImhQ4fyzjvv4GNRb3vQQoLKQ/Zt3Mh3752hQc5i\n/vjhK1DCL+2q/ae575MN1AvyY/qdvWnfNKQSI60etJBg2aSlpVGvXj1sNhsjR47k/vvvz7+GMGLE\nCN58802io6PzlwN48cUXSUlJ4fXXX/dm6AC8+uqrNGnShPHjx1v6PhUpJKhHFsoj1k7/HsSHnlfW\nKzFRfLPlGHdMXc9loUF8eX9/TRTKI5555hl69OhBt27d6NChQ5ElygEWLFhAbGwsMTExrF69miee\neMJbIRegJcorkR5ZeE+OzcbH987D136aO9/9A9RtXORyH/9ykEnf7KBXqzA+vD2e0DrujZaqjfTI\nQllBS5Qrr1rx6QyyAyOIqPdTkYnCGMPLi3fz3or9XNelKW/d0oMgf18vRKqUKi9NFqrCjvx8Gl+f\nMAaNH3HJPJs9l8e+3MLcX5MY1zeS/zciBl+fmtmwSKmaTJOFqpCTRw6R4deFerbV1I95rsC8i1k5\n3P/5r6zck8zfrm3Pg4Pa1tjOdkrVdJosVIWsfGcGxqcPHfsIuCSC02lZ3DltPduPneflm7syplek\nF6NUSlWUjoZSFZJ6ojmBGUfofccj+dMOn7nI6HdXsefkBT64racmimpOS5Rbz7VE+YULFxg6dCgd\nOnSgS5cuPPXUU8WuV5klyvXIQpXbuvlzyQpqSXjAtxB0BwBbE1OZMG0d9lzD/+7pS1xkQ+8GqSos\nr6DetGnTSEhI4L///W+5tjN37lx8fHzo2LGjJ8MrUV6J8l9//bXS3rM88kqUv/vuu4gIjz32GFde\neSVZWVkMGjSIJUuWcM011xRYx7VE+dGjRxkyZAi7d++27MY8PbJQ5bbrm91Iro0rx18LwE97k7nl\ng9UE+vky5/7+mihqAS1R7vgcnixRXq9ePa688krAUW+qR48eJCYmXrIvtES5qhYunDtLuokhOHsz\nTXv+na82JvG3LzbTrmkI0yb0omn9IG+HWGP8NHsPp4+meXSbjVvW4/I/tK/QNrREufUlys+ePcui\nRYv4+9//fsn+qFElykVkiIjsFpF9IvJ4EfPfEJFNzsceETnnMm+8iOx1Pqy9B16V2bJ/v4fdry4t\nYzL4cOUB/jxrE72iwph1X19NFLWElii3tkS5zWZjzJgx/PWvf6VVq0uLbNaYEuUi4gu8DVwDJALr\nRWSBMSZ/7xhjHnFZ/k9AD+fzMOA5IB4wwAbnumetileVzen9DQjwOc26NkP4YNFOhnVrxr/+0J1A\nP73ZztMqegRgFS1Rbl2JcmNMfvJ66KGHioy5JpUo7w3sM8YcMMZkAzOBkSUsPxaY4Xx+HbDEGJPi\nTBBLgCEWxqrKYPfKH8gIaoePzyY+WHeaO/pH8Z9bemiiqGW0RHnZlKVE+RNPPEFmZmb+Ka6iVHaJ\nciuvWbQAjrq8TgT6FLWgiLQCWgM/lrBuCwtiVOWwfsZPwOUsqR/K49d35L4rovVmu1rItUR5dnY2\nAJMnTyY4OJhRo0aRlZVFbm5ugRLl9913H6+//jpfffUVUVFRZXo/1xLlxhiGDx/OsGHDAPJLlLdo\n0eKSEuWuF5jzSpSHhIRwxRVXFChRPnr0aGbMmMHVV19teYny6OjoYkuUHzp0iJdffplOnToRFxcH\nwMMPP8yECROYN28eW7du5dlnny1QotzPz6/6ligXkd8D1xlj7na+vg3obYz5UxHLPgZE5M0TkUeB\nQGPMC87XzwDpxpjXC613L3AvQGRkZM/Dhw+XOc59mxNY/sbuMq9Xm9n8wwjIOkDDP49kVFzpTY5U\n2WkhwbLREuXuqaqFBBOBli6vI4DiGszeAjxYaN2BhdZdXnglY8wHwAfgqDpbniD9AoMQc6I8q9Za\nAdknCOvlx02aKFQV8cwzz7B8+XIyMzMZMmRIkSXKo6OjWbBgAa+88go5OTlERUUxbdo07wXtolaX\nKBcRP2APcBWQBKwHbjXGbC+0XAfgO6C1cQbjvMC9AYhzLvYr0NMYk1Lc+2mJclWT6JGFskKVPLIw\nxuSIyEM4EoEvMNUYs11EJgEJxpgFzkXHAjONS9YyxqSIyD9wJBiASSUlCqWUUtay9KY8Y8wiYFGh\nac8Wev18MetOBaZaFpxSVZwxRgcOKI+p6FkkLfehVBUUFBTEmTNnLBnCqWofYwxnzpwhKKj8N8xq\nuQ+lqqCIiAgSExNJTk72diiqhggKCiIiovyDUjRZKFUF+fv707p1a2+HoVQ+PQ2llFKqVJoslFJK\nlUqThVJKqVJZdlNeZRORZKDs9T5+0xg47aFwPEnjKhuNq2w0rrKpiXG1MsaEl7ZQjUkWFSUiCe7c\nxVjZNK6y0bjKRuMqm9ocl56GUkopVSpNFkoppUqlyeI3H3g7gGJoXGWjcZWNxlU2tTYuvWahlFKq\nVHpkoZRSqlS1NlmIyKsisktEtojIPBFpUMxyQ0Rkt4jsE5HHKyGu34vIdhHJFZFiRzeIyCER2Soi\nm0TE8kYeZYirsvdXmIgsEZG9zp8Ni1nO7txXm0RkQVHLeCieEj+/iASKyCzn/LUiEmVVLGWM6w4R\nSXbZR3dXQkxTReSUiGwrZr6IyL+dMW8RkbiilvNCXANFJNVlXxXfONyzcbUUkWUistP5f/HhIpax\nbp8ZY2rlA7gW8HM+fxl4uYhlfIH9QDQQAGwGOlscVyegA47OgPElLHcIaFyJ+6vUuLy0v14BHnc+\nf7yof0fnvLRK2Eelfn7gAeA95/NbgFlVJK47gP9W1u+T8z2vwNHgbFsx84cC3wIC9AXWVpG4BgLf\nVOa+cr5vMyDO+TwER3O5wv+Olu2zWntkYYz53hiT43y5Bkfr1sJ6A/uMMQeMMdnATGCkxXHtNMZU\nuabgbsZV6fvLuf3pzufTgRstfr+SuPP5XeOdA1wl1jet8Ma/S6mMMSuBkpqajQQ+MQ5rgAYi0qwK\nxOUVxpjjxphfnc8vADuBFoUWs2yf1dpkUcidOLJxYS2Aoy6vE7n0H8dbDPC9iGwQkXu9HYyTN/ZX\nU2PMcXD8ZwKaFLNckIgkiMgaEbEqobjz+fOXcX5ZSQUaWRRPWeICuNl56mKOiLS0OCZ3VOX/f/1E\nZLOIfCsiXSr7zZ2nL2sxVk0AAAS5SURBVHsAawvNsmyf1egS5SLyA3BZEbOeMsbMdy7zFJADfF7U\nJoqYVuHhY+7E5YYBxphjItIEWCIiu5zfiLwZV6XvrzJsJtK5v6KBH0VkqzFmf0VjK8Sdz2/JPiqF\nO+/5NTDDGJMlIhNxHP0Mtjiu0nhjX7njVxwlMtJEZCjwFdCust5cROoBXwJ/NsacLzy7iFU8ss9q\ndLIwxlxd0nwRGQ/cAFxlnCf8CkkEXL9hRQDHrI7LzW0cc/48JSLzcJxqqFCy8EBclb6/ROSkiDQz\nxhx3Hm6fKmYbefvrgIgsx/GtzNPJwp3Pn7dMooj4AaFYf8qj1LiMMWdcXn6I4zqet1ny+1RRrn+g\njTGLROQdEWlsjLG8ZpSI+ONIFJ8bY+YWsYhl+6zWnoYSkSHAY8AIY0x6MYutB9qJSGsRCcBxQdKy\nkTTuEpG6IhKS9xzHxfoiR25UMm/srwXAeOfz8cAlR0Ai0lBEAp3PGwMDgB0WxOLO53eNdzTwYzFf\nVCo1rkLntUfgOB/ubQuA250jfPoCqXmnHL1JRC7Lu84kIr1x/B09U/JaHnlfAT4Cdhpj/lXMYtbt\ns8q+ol9VHsA+HOf2NjkfeSNUmgOLXJYbimPUwX4cp2OsjusmHN8OsoCTwHeF48IxqmWz87G9qsTl\npf3VCFgK7HX+DHNOjwemOJ/3B7Y699dW4C4L47nk8wOTcHwpAQgCvnD+/q0Doq3eR27G9U/n79Jm\nYBnQsRJimgEcB2zO3627gInAROd8Ad52xryVEkYHVnJcD7nsqzVA/0qK63c4Tiltcfm7NbSy9pne\nwa2UUqpUtfY0lFJKKfdpslBKKVUqTRZKKaVKpclCKaVUqTRZKKWUKpUmC6XKQETSKrj+HOdd5IhI\nPRF5X0T2O6uIrhSRPiIS4Hxeo2+aVdWLJgulKomzhpCvMeaAc9IUHHdvtzPGdMFR+bWxcRT7WwqM\n8UqgShVBk4VS5eC8Q/ZVEdkmjr4iY5zTfZzlH7aLyDcisuj/t3fHqlGEURiG36OgCCmEKGInGKsg\nsRGLTeMVCCJYpPUaAgFBJFdh4w2ImsJYCgYrtVAMpLGyiqSyiijJsTj/gBa7E4aNLvg+1Rb7DzPF\n7uHf2fm+iLjTlq3QnjCPiMvADeB+Zh5CRZFk5mZ770Z7vzQT3OZKw9wGrgFLwDngXURsUVEil4Cr\nVALuDvC4rRlRTwcDLAIfMvNgzPG3gevHcubSAO4spGGWqZTWg8z8CrymvtyXgSeZeZiZu1R0Ruci\nsHeUg7ch8qPLAJP+NYeFNMy4wqJJRUb7VDYUVLbQUkRM+gyeBr4PODdp6hwW0jBbwN2IOBkR56kq\nzrfAG6pE6EREXKAqODs7wAJAVpfGe+DhbwmmVyLiVns9D+xl5s+/dUHSJA4LaZjnVPrnR+AVsNp+\ndnpKJZVuA4+oJrNvbc0mfw6Pe1Sp0+eI+ET1SHTdAzeBl8d7CdLRmTorTVlEzGW1qM1Tu41RZu5G\nxBnqHsZowo3t7hjPgLWcwT52/Z/8N5Q0fS8i4ixwClhvOw4ycz8iHlCdyF/GLW4FRRsOCs0SdxaS\npF7es5Ak9XJYSJJ6OSwkSb0cFpKkXg4LSVIvh4Ukqdcv4UvWGROKSLAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21f11257978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "accuracy_s1 =np.array(accuracy_s).reshape(len(C_s),len(gamma_s))\n",
    "x_axis = np.log10(C_s)\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis, np.array(accuracy_s1[:,j]), label = ' Test - log(gamma)' + str(np.log10(gamma)))\n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "plt.savefig('SVM_RBF_RentListing.png' )\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  从结果中可得 C=1，gamma=100时 最佳"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 5. 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7142618, 7210040, 7103890, ..., 6882352, 6884758, 6924212], dtype=int64)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "listing_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "submission = pd.DataFrame()\n",
    "submission['Id'] = listing_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "SVC_last =  SVC( C = 1, kernel='rbf', gamma = 100)\n",
    "SVC_last = SVC_last.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "prediction = SVC_last.predict(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "submission['interest_level'] = prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "_map = {0: 'low', 1: 'medium', 2:'high'}\n",
    "submission['interest_level'] = submission['interest_level'].apply(lambda y: x_map[y])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "submission.to_csv('submission_svm.csv',index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
